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INTRODUCTION 


Over  1.2  million  people  in  America  have  missing  limbs  resulting  from  combat  (e.g.,  wars  and  other  conflicts)  and 
non-combat  (e.g.,  accidents,  birth  defects)  operations.  Thus,  availability  of  artificial  limbs  will  help  these  people  to 
lead  a  more  normal  life.  Although  artificial  devices  have  been  around  for  the  last  30  years,  there  are  no  artificial 
hands  that  fully  simulate  the  various  natural/human-like  operations  of  moving,  grasping,  lifting,  and  twisting. 
Moreover,  prosthetic  devices  still  pose  significant  biocompatibility  problems,  that  need  to  be  addressed  and  tissues 
surrounding  implants  develop  inflammatory  reactions”. 

The  present  Phase  II  of  Smart  Prosthetic  Hand  Technology  focuses  on  the  four  closely  connected  areas  of  EMG 
signal  identification  and  estimation,  hand  motion  estimation,  intelligent  embedded  systems  and  control,  robotic  hand 
and  biocompatibility  and  signaling.  The  developed  identification  algorithm  using  a  new  sensor  array,  a  proposed 
hybrid  estimation  algorithm  will  provide  for  the  decomposition  and  inference  of  multiple  signals.  Different 
categories  of  amputation  level  will  be  simulated  and  the  identification  of  EMG  signals  will  be  tested  and  adapted  to 
these  different  cases.  Next,  the  proposed  sensing  and  actuation  system  for  the  artificial  hand  overcomes  the 
limitations  on  dexterous  manipulation  of  the  prosthetic  hands.  Strategies  on  grasping  and  manipulation  will  be 
developed  to  complement  the  myoelectric  signals.  The  proposed  fusion  of  soft  and  hard  embedded  control  systems 
strategy  will  alleviate  the  present  problems  associated  with  prosthetic  devices.  Finally,  by  investigating  into  the 
issues  of  the  inflammatory  responses  of  cells/tissues  in  response  to  an  artificial  implant  and  the  interference  with 
signaling  of  the  artificial  implant,  the  design  of  our  in-vitro  model  will  ultimately  improve  the  design  and  construct  a 
functional  and  biocompatible  artificial  limb. 


BODY 


GOAL  1:  EMG  Signal  Identification: 

1.  Overview 

Prosthesis  can  be  greatly  useful  for  the  amputees  which  can  impact  their  life  in  a  positive  way.  The  objective  of 
this  research  is  to  design  a  =Smart  Prosthetic  Hand4  for  the  people  with  upper  extremity  amputations.  In  control  part 
of  the  prosthesis  we  have  two  main  objectives,  force  and  position  control.  Therefore,  to  have  an  effective  control  of 
force  and  position  we  need  to  have  the  information  of  force  and  positions  in  advance.  This  makes  it  absolutely 
necessary  to  have  an  estimation  scheme  or  model  in  place  that  can  compute  the  required  position  and  force  to 
perform  a  specific  task  of  grasping  or  manipulation.  There  has  been  active  research  in  past  towards  prosthetic  hand 
design  that  have  similar  functionality  and  appearance  as  human  hands  [1,  2].  Surface  electromyographic  (sEMG) 
signals  are  the  sole  control  input  for  most  of  these  research  works  where  the  recorded  sEMG  signal  is  used  as  an 
input  to  activate  the  prosthesis.  sEMG  signals  depend  on  the  flow  of  specific  ions  including  sodium  (Ar&+), 
potassium  (K+)  and  calcium  (Ca++)  resulting  in  the  action  potentials  in  nerves  and  their  respective  skeletal  muscle 
fibres  which  get  activated  and  controlled  by  the  central  nervous  system.  These  ion  exchanges  cause  a  potential 
difference  across  neuronal  membranes  which  can  be  measured  as  an  electrical  voltage  change  [3].  An  EMG  signal 
recorded  on  the  surface  of  the  limb  is  expressed  as  an  electric  voltage  ranging,  between  -5  and  +5  mV.  The 
movement  and  force  control  of  the  prosthesis  in  [4,  5]  uses  sEMG  signal  as  an  input  to  the  controller.  Past  research 
results  show  that  EMG  signal  amplitude  generally  increases  with  skeletal  muscle  force.  However,  this  relationship  is 
not  always  same;  as  there  are  other  factors  that  influence  the  muscular  force.  There  are  numerous  factors  such  as  the 
varying  motor  unit  recruitments,  crosstalk,  and  biochemical  interaction  within  the  muscular  fibres  that  affect  the 
sEMG  signals.  This  makes  EMG  signals  random,  complex,  and  dynamic  in  nature  and  the  control  of  the  prosthesis 
difficult.  Moreover,  it  changes  continuously  due  to  the  onset  and  progression  of  muscle  fatigue  which  results 
because  of  continuous  high  frequency  stimulation  or  because  of  tetanic  stimulation  [6].  Synchronization  of  active 
motor  units  along  the  muscle  fibres,  and  a  decrease  in  conduction  velocity  are  reflected  in  the  EMG  signal  as  an 
increase  of  amplitude  in  the  time  domain  and  a  decrease  of  medium  frequency  in  the  frequency  domain  [6].  All 
these  factors  make  the  relationship  between  EMG  and  force  nonlinear.  Correct  interpretation  of  EMG  signal  is  vital 
to  achieve  precise  motion  and  force  control  of  prosthesis.  EMG  analysis  is  a  well-accepted  method  for  muscle 
fatigue  assessment  [7]. 

In  1977,  L.  Lindstrom,  R.  Kadefors  and  I.  Petersen  developed  a  method  that  measures  the  localized  muscle 
fatigue  based  on  the  power  spectrum  analysis  using  myoelectric  signals  [8].  This  approach  permits  real-time 
investigations  and  can  yield  statistically  based  criteria  for  the  occurrence  of  fatigue.  Rate  of  fatigue  development  and 
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changes  in  muscle  action  potential  conduction  velocity  were  used  to  interpret  the  findings  in  [8].  The  joint  analysis 
method  using  EMG  amplitude  and  spectrum  (JASA)  allows  distinguishing  between  the  difference  of  fatigue -induced 
and  force  related  EMG  changes.  Simultaneous  changes  in  the  EMG  amplitude  and  spectrum  are  considered  in  the 
JASA  approach.  The  JASA  principle  states  that  muscle  force  decreases  because  of  the  decrease  in  EMG  amplitude. 
Fatigue  can  occur  because  of  continuous  high  frequency  stimulation  or  because  of  tetanic  stimulation. 

This  part  of  the  research  addresses  issues  with  the  position  and  force  estimation  and  their  variations  due  to  the 
muscle  fatigue.  The  sEMG,  skeletal  muscle  force  and  the  angles  of  the  proximal  interphalangeal  (PIP)  joint  of  the 
index  finger  signals  are  filtered  using  different  filters  and  modeled  with  linear  and  nonlinear  System  Identification 
techniques  and  adaptive  neuro-fuzzy  inference  (ANFIS)  systems.  The  outputs  of  obtained  models  are  fused  using  an 
adaptive  probability  of  Kullback  Information  Criterion  (KIC)  for  model  selection  to  achieve  better  force  and  angle 
estimates.  Following  part  of  the  report  presents  Experimental  Set-Up  and  Signal  Pre-Processing,4  =Skdetal  Muscle 
Fatigue  and  Spectral  Analysis  of  sEMG  Signals,4  J3ifferent  Modeling  Approaches,4  JTda  Fusion  Algorithm,4 
Jesuits  and  Discussion,4  and  =C°ndusi°n  and  Future  Work.4 


2.  Classification  of  amputation 

Individuals  with  amputations  of  either  the  upper  or  lower  extremities  are  generally  classified  by  the  level  of  the 
amputation  in  relation  to  the  most  proximal  joint.  For  example,  a  person  having  the  hand  amputated  would  be 
considered  a  trans-radial/ulnar  amputee  (below  the  elbow).  If  the  amputation  was  through  the  humerus,  this  would 
be  a  trans-humeral  (above  the  elbow)  However,  this  classification  only  grossly  defines  amputations  and  makes  no 
reference  to  the  true  length  of  the  residual  limb  or  the  specific  musculature  that  remains  after  appropriate  surgical 
intervention.  In  line  with  our  studies  investigating  sEMG  signal  identification  and  filtering,  fatigue  estimation  and 
array  sensor  development,  we  have  completed  the  measurements  of  specific  muscle  lengths  of  16  fully  dissected 
normal  cadaver  arms  to  determine  the  most  appropriate  and  functional  residuum  length  giving  the  best  possible 
potential  for  sEMG  signaling  from  the  remaining  musculature  post  amputation.  It  is  well  known  that  surgeons  have 
the  task  of  making  decisions  related  to  level  of  amputation  in  both  traumatic  and  planned  amputations  and  the 
general  consensus  is  that  the  longer  the  residuum  and  proximal  joint  sparing,  the  better  the  potential  for  prosthetic 
function  [9].  There  is  also  the  concern  of  muscle  atrophy  in  limbs  post  amputation  which  may  affect  the  potential 
for  socket  fit  and  myoelectric  prosthetic  use.  The  greater  length  of  the  residuum  has  clear  functional  benefits 
especially  when  the  musculature  the  limb  is  relatively  intact  and  has  enough  length  to  maintain  motor  point 
locations. 

2.1  Physical  classification 

The  arms  of  three  female  and  four  male  cadaver  specimens  (16  arms)  were  procured  from  the  existing  cadaver 
holdings  at  Idaho  State  University.  The  mean  height  of  the  specimens  was  171.175  cm  (SD  9.114).  Cadavers  were 
neatly  dissected  with  all  superficial  skin,  adipose  and  fascia  removed  from  the  upper  extremities  and  shoulder  girdle. 
Measurements  were  taken  by  three  researchers  with  an  inter-tester  reliability  of  r=.866  using  both  200mm  and  300 
mm  Vernier  Aerospace®  calipers  as  seen  in  Fig.  2.1.1.  The  measurements  of  the  upper  extremity  musculature 
included  the  distance  from  the  acromioclavicular  (AC)  joint  to  the  medial  and  lateral  condyles  of  the  humerus,  the 
distance  from  the  medial  humeral  condyle  to  the  distal  myotendinous  (MTJ)  junction  of  the  majority  muscles  of  the 
flexor  compartment  of  the  forearm,  and  the  distance  from  the  lateral  humeral  condyle  to  the  distal  MTJ  of  the 
majority  muscles  in  the  extensor  compartment  of  the  forearm  (see  Fig.  2.1.2).  In  addition,  all  muscle  length 
measurements  of  muscles  of  the  upper  arm  on  both  posterior  and  anterior  regions  were  taken  from  the  AC  joint  to 
the  distal  MTJ.  The  muscles  measured  in  the  forearm  are  considered  extrinsic  movers  of  the  hand  and  fingers.  The 
muscles  in  the  upper  arm  are  considered  movers  of  the  elbow  and,  in  part,  the  shoulder  girdle.  The  measurement  for 
the  forearm  and  upper  arm  musculature  in  both  the  flexor  and  extensor  compartments  were  analyzed  for  means,  and 
standard  deviations  and  ratios  of  the  measurements  to  the  mean  AC  to  medical  and  epicondyle  values.  Table  2.1.1 
includes  the  summary  of  measurements  from  the  AC  joint  to  the  medial  and  lateral  epicondyles.  Table  2.1.2,  2.1.3 
and  2.1.4  includes  muscles  which  were  measured  at  each  compartment  of  the  entire  arm. 
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Fig:  2.1.2.  Example  of  Measurement  Taken  on  the  Extensor  Carpi  Ulnaris  Muscle  in  Forearm  of  Extensor 

Compartment. 


Fig:  2.1.3.  Example  of  Measurement  of  Deltoid  Muscle  in  Upper  Arm. 

TABLE  2.1.1 

MEASUREMENT  OF  ACROMIOCLAVICULAR  JOINT  TO  MEDIAL  AND  LATERAL  EPICONDYLES 


Mean 

SD 

Medial  Condyle 

31.780  cm 

2.1 18  cm 

Lateral  Epicondyle 

32.638  cm 

2.205  cm 

Combined 

32.209  cm 

2.165  cm 
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TABLE  2.1.2 

MUSCLE  MEAN  MEASUREMENTS  OF  FOREARM  AND  UPPER  ARM 
EXTENSOR  GROUPING 


Muscle 

Mean  (cm) 

SD  (cm) 

Brachioradialis 

17.873 

3.221 

Extensor  carpi  radialis  longus 

18.477 

2.650 

Extensor  carpi  radialis  brevis 

15.683 

2.205 

Extensor  digitiorum 

18.306 

2.598 

Extensor  digitorum  minimi 

16.776 

2.705 

Extensor  carpi  ulnaris 

17.820 

2.854 

Anconeus 

6.960 

1.872 

Supinator 

6.856 

1.598 

Abductor  policis  longus 

15.545 

2.111 

Extensor  policis  longus 

13.820 

1.896 

Extensor  policis  brevis 

7.631 

1.601 

TABLE  2.1.3 

MUSCLE  MEAN  MEASUREMENTS  OF  FOREARM  AND  UPPER  ARM 
FLEXOR  GROUPING 


Muscle 

Mean  (cm) 

SD  (cm) 

Pronator  teres 

15.040 

1.705 

Flexor  carpi  radialis 

17.942 

3.050 

Flexor  carpi  ulnaris 

21.030 

3.339 

Flexor  digitorum  superficialis 

20.726 

3.201 

Flexor  digitorum  profundus 

18.303 

1.211 

Flexor  policis  longus 

15.438 

2.063 

TABLE  2.1.4 

MUSCLE  MEAN  MEASUREMENTS  OF  FOREARM  AND  UPPER  ARM 
UPPER  ARM 


Muscle 

Mean  (cm) 

SD  (cm) 

Biceps  brachii  long  head 

25.821 

2.603 

Biceps  brachii  short  head 

23.832 

3.126 

Brachialis 

24.765 

2.652 

Deltoid 

16.272 

2.212 

Triceps  long  head 

22.578 

3.226 

Triceps  lateral  head 

16.671 

2.716 

Ratios  of  the  mean  muscle  length  values  to  the  length  of  the  mean  AC  to  mean  lateral  and  medial  epicondyle 
measures  were  calculated  to  ascertain  the  level  of  the  distal  residuum  where  the  majority  of  muscles  and  respective 
motor  points  would  be  located  to  enhance  the  potential  for  sEMG  signal  monitoring  in  individuals  with  amputations. 
The  calculated  ratios  for  the  forearm  musculature  in  both  flexor  and  extensor  groupings  was  .636  ±  .0535.  The  ratio 
of  the  mean  upper  arm  musculature  length  to  the  AC  condylar  distance  is  .505  ±  .041.  Given  these  ratios  of  length 
to  a  known  distance,  the  measurements  suggest  that  residuum  arms  that  are  distal  (Zone  1)  to  the  ratio 
measurement  would  be  ideal  for  sEMG  signaling  since  they  would  have  almost  all  the  forearm  or  upper  arm 
musculature  intact.  If  the  residuum  was  proximal  (Zone  2)  to  the  ratio  measurement  it  would  still  be  appropriate  for 
sEMG  signaling  since  the  motor  points  of  the  major  movers  of  the  hand  would  still  be  intact  allowing  for  adequate 
signaling.  Although  individual  variation  will  occur,  motor  point  mapping  of  specific  musculature  has  been 
documented  [10].  However,  residuum  arm  lengths  that  would  be  much  shorter,  would  present  some  difficulty  for 
detecting  sEMG  signaling.  As  an  example,  Fig.  2.1.4  and  2.1.5  show  one  of  the  cadaver  specimens  on  which  the 
Zones  have  been  depicted.  Arrows  depict  the  Zone  measurements  as  calculated  per  the  derived  ratios.  The  AC  to 
mean  condylar  measurement  of  this  specimen  is  30.5  cm.  In  both  cases,  the  demarcation  of  Zone  2  suggests  that  the 
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number  of  muscles  and  their  motor  points  would  be  adequate  for  harvesting  the  sEMG  signal  in  respective  upper 
arm  and  forearm  musculature.  In  Fig.  2.1.4  the  demarcation  of  Zone  1  and  2  is  measured  at  19.39  cm  distal  to  the 
elbow  joint.  In  Fig.  2.1.5  the  demarcation  of  the  Zones  is  at  15.40  cm  distal  to  the  AC  joint.  In  both  cases  it  is 
evident  that  the  bulk  of  the  musculature  is  located  proximal  to  the  line  of  Zone  delineation. 


Fig.  2.1.4.  Zone  Delineation  on  Forearm  Musculature. 


Fig:  2.1.5.  Zone  Delineation  on  Upper  Arm. 


With  the  development  of  this  classification  and  the  measurements  of  the  distal  Zones,  we  plan  to  move  ahead  in 
this  next  quarter  with  sEMG  signaling  from  Zone  1  a  2  in  the  forearm  to  validate  this  amputation  classification 
sEMG  signaling.  In  addition,  the  array  sensor  experiments  using  this  classification  will  also  investigate  the 
parameters  of  signal  filtering  and  fatigue  in  the  musculature  in  these  zones.  .  Further,  sEMG  signaling  of  the  arm 
musculature  during  the  movements  of  forearm  pronation  and  supination  and  force  data  in  various  normal  subjects 
will  also  be  tested. 


2.2  Experimental  set-up  and  spatial  filtering/signal  pre-processing 
2.2.1  Spatial  Filtering: 

’’Spatial  filtering”  is  broadly  defined  as  methods  which  compute  spatial  density  estimates  for  events  that  have 
been  observed  at  individual  locations.  Spatial  filtering  describes  a  set  of  tools  for  displaying  functions  estimated 
from  these  data  points  which  are  generally  distributed  in  two-dimensional  space.  Spatial  filtering  can  also  be  thought 
of  as  a  form  of  data  smoothing  method  which  is  designed  to  give  us  a  clearer  view  of  the  general  underlying 
information. 

Spatial  filtering  is  a  non-parametric  analysis  method  which  belongs  within  the  field  of  exploratory  spatial 
analysis  which  relies,  to  a  large  degree,  on  graphical  methods  of  analysis.  Spatial  filters  are  used  when  there  is  no  a 
priori  curve  to  fit  to  a  data  series.  Instead,  it  relies  on  nearby  or  adjacent,  values  to  estimate  the  value  at  a  given 
point.  These  filters  take  out  variability  in  a  data  set  while  retaining  the  local  features  of  data.  By  varying  the  size  of 
the  filter,  features  in  the  data  that  vary  at  different  spatial  scales  can  be  differentially  removed.  Spatial  filtering  is 
useful  as  an  exploratory  technique  for  identifying  areas  that  are  homogeneous  or  areas  that  have  larger  or  smaller 
values  than  which  generally  occurs. 
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Spatial  filtering  is  principally  associated  with  digital  image  processing.  This  method  may  be  applied  to  almost 
any  type  of  grid/data  set  or  image  (which  contains  the  gray  scale  values  or  pixel  data  in  the  form  of  a  grid).  This 
term  can  also  be  used,  in  a  related  manner,  in  the  area  of  spatial  statistics.  The  most  commonly  provided  functions  of 
spatial  filtering  are  the  so-called  low-pass  and  high-pass  spatial  filters.  These  are  focal  functions  whose  operation  is 
determined  by  a  kernel  or  neighborhood  of  NxN cells  around  each  pixel  or  grid  position  [11].  Grid  cells  -covered” 
by  a  kernel  are  multiplied  by  the  matching  kernel  entry  and  then  the  weighted  average  is  calculated  and  assigned  as 
the  value  for  the  central  cell,  G.  For  example,  asymmetric  3x3  kernel  may  look  like  the  one  shown  in  (1),  or  any 
combination  of  the  weights.  Typically  a,  b  are  positive  integers.  If  a=b=  1,  then  the  kernel  provides  a  simple 
smoothing  or  averaging  operation.  The  weights  in  the  kernel  can  be  modified  for  specific  cases  or  data  sets.  In  any 
case  the  weighted  average  is  divided  by  the  sum  of  the  elements  of  the  kernel.  Filters  of  this  type  are  sometimes 
referred  to  as  low-pass  filters. 


Symmetric  Kernel  = 


a 

a 

a 

a 

b 

a 

a 

a 

a 

(1) 


If  the  weights  in  the  kernel  looks  like  the  one  in  (2)  and  a,  b,  and  c  are  positive  integers. 

c  a  c 


Symmetric  Kernel  with  different  weights  = 


a 


a 


(2) 


And  if  the  following,  b>a>c ,  is  true  then  the  kernel  is  described  as  a  Gaussian  filter  which  is  symmetric  but  center- 
weighted. 

The  filtered  grid  value  =G"of  an  m=NxN  kernel  matrix,  with  Q  set  of  coefficients  and  Pt  -  set  of  source  grid 
values,  is  calculated  as; 


G  = 


(3) 


i= l 

Where,  B  is  often  set  to  0.  B  is  a  bias  term  to  increase  or  decrease  the  resulting  value  of  jG" 

There  are  many  different  forms  of  spatial  filters  depending  on  the  weights  and  the  symmetry  associated  with  the 
kernel.  This  kernel  is  also  sometimes  referred  to  as  the  ,  filter  mask  "  Some  of  the  other  examples  are  given  in  Table 
2.1.1. 


Linear  Spatial  Filtering: 

Linear  spatial  filtering  modifies  an  image  J^by  replacing  the  value  at  each  pixel  with  a  linear  function  of  the 
values  of  nearby  pixels  (This  same  analogy  can  be  applied  to  data  set  in  a  grid  format;  similar  to  the  case  of  multiple 
sEMG  sensors  placed  in  grid).  Moreover,  this  linear  function  is  assumed  to  be  independent  of  the  pixel’s  location  ( k , 
/),  where  ( k ,  I)  are  the  indices  of  the  pixels  in  f  which  is  represented  by  a  data  matrix.  This  kind  of  operation  can  be 
expressed  as  convolution  or  correlation.  For  spatial  filtering,  it’s  often  more  intuitive  to  work  with  correlation. 

The  filtered  result  g(k,  1)  is  obtained  by  centering  the  mask  over  pixel  (k,  /)  and  multiplying  the  elements  of / 
with  the  overlapping  elements  of  the  mask  and  then  adding  them  up.  A  special  case  exists  when  the  special  when  the 
center  of  the  mask  is  on  the  boundary  of  f  The  common  assumption  in  this  case  is  that  the  data  matrix  is  periodic. 
But  this  assumption  does  not  always  hold  true.  In  such  cases  the  non-existing  entries  assumed  to  be  zero. 
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TABLE  2.2.1 

LINEAR  SPATIAL  FILTERS 


Filter  Type  Filter  Kernel  Description 


Averaging 

Low-pass 

a  a  a 

aba 

a  a  a 

’ 

0  a  0 

aba 

0  a  0_ 

Smoothing,  Noise 
Reduction  or  Blurring 
Filter  (focal  mean) 

(symmetric) 

Gaussian 

c  a  c 

aba 

c  a  c 

Smoothing,  Noise 
Reduction  or  Blurring 
Filter  (focal  weighted 
mean) 

High-pass 

,  .  4  Sharpening 

(symmetric) 

—a  —a  —a 

-a  +b  -a 

-a  -a  -a 

- 

0  a  0 

-a  b  -a 

0  -a  0 

Mean  effect 

removal/ sharpening 
filter  (focal  sum)  . 

Provides  limited  edge 

detection.  Typically 

entries  sum  to  1  but 
may  be  greater. 

Edge  Detection 

aba 

0  0  0 

-a  -b  -a 

? 

-a  0  a 

-b  0  b 

-a  0  a 

Kernel  highlights 
vertical  and  horizontal 
edges.  Typically  <3=1, 
and  b=l  or  2  and 

entries  sum  to  0 . 

Embossing 

Gradient 

/  — »  o  t  7mm  +-  -i'-'  -i  n  \ 

0  -\-a  -\-a 

-a  +a  +a 

-a  -a  0 

Enhance  edges  in  a 
selected  direction  to 
provide  embossed 
effect . 

Directional 

"-1  1  1" 

-1  -2  1 

-1  1  1 

"  1  2  1 

0  0  0 

-1  -2  - 

1 

"  i  i  r 

1  -2  1 

-i  -i  -i 

Application  of  Spatial  Filters  to  sEMG  data  Array: 

Recording  and  analysis  of  Surface  EMG  (sEMG)  has  been  around  for  almost  a  century  now.  Surface  electrodes 
have  been  routinely  used  to  record  the  gross  electrical  activity  of  skeletal  muscles.  However,  when  specific 
information  about  individual  motor  unit  discharge  rates  or  recruitment  needs  to  be  extracted,  sEMG  signal  have  been 
treated  with  some  reservation.  So  previously,  in  most  cases,  intramuscular  EMG  recording  using  fine  wire  or  needle 
electrodes  were  employed  to  selectively  record  the  single  motor  unit  activity.  Unfortunately,  invasive  recordings  of 
sEMG  are  inconvenient,  do  induce  pain  to  the  subject  and  may  potentially  damage  muscle  tissues  and  nerves.  In 
recent  years,  considerable  efforts  have  been  directed  towards  effectively  recording  single  motor  unit  activity 
noninvasively  i.e.  using  surface  EMG  sensors. 

The  principle  underlying  this  approach  is  to  be  increasingly  selective  in  recording  the  information  using  surface 
EMG  electrodes.  In  other  words,  the  objective  is  to  amplify  the  activity  of  motor  unit/s  located  closest  to  the 
recording  site  (generally  the  actual  motor  location  for  the  particular  limb)  and  reducing  the  EMG  signal  generated  by 
other  motor  units  located  further  away.  The  selectivity  of  surface  EMG  recordings  can  be  increased  by  reducing  the 
electrode  size  (i.e.,  skin-electrode  contact  area  or  inter-electrode  distance)  [12,  13]  and/or  by  applying  temporal 
filters  [14].  More  recent  work  has  focused  on  recent  advances  in  the  design  of  surface  electrode  arrays  [15-17]  to 
extract  single  motor  unit  information  from  sEMG.  A  large  number  of  traditional  [18-21]  and  adaptive  [22,  23]  linear 
spatial  filters  have  been  extensively  used  to  glean  more  information  out  of  sEMG  signals  and  to  understand  it  much 
better. 

Many  types  of  nonlinear  spatial  filters  have  also  been  studied  in  the  past.  In  this  study  comparison  of  a  few 
spatial  filters  is  presented. 
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Experiments  were  performed  using  multiple  sEMG  sensors  in  the  array  configuration  as  shown  in  Fig.  2.2.1.  Nine 
different  experiments  were  conducted  and  the  corresponding  sEMG  was  measured  simultaneously  from  all  the 
different  sensors.  Information  regarding  the  various  experiments  is  given  below: 

Experiment  1  -  Baseline  -  30  sec  -  No  Motion/  Force  -  Red  Stress  ball 
Experiment  2  -  Baseline  -  30  sec  -  No  Motion/  Force  -  Red  Stress  ball 
Experiment  3  -  30  sec  -  Ring  Finger  Grasping  Motion  -  Red  Stress  ball 
Experiment  4  -  30  sec  -  Ring  Finger  Grasping  Motion  -  Red  Stress  ball 

Experiment  5  -  45  sec  -  Ring  Finger  Grasping  Motion  with  Thumb  Restrained  -  Red  Stress  ball 
Experiment  6  -  45  sec  -  Ring  Finger  Grasping  Motion  with  Thumb  Restrained  -  Red  Stress  ball 
Experiment  7  -  45  sec  -  Ring  Finger  Grasping  Motion  with  Thumb  Restrained  -  Yellow  Stress  ball 
Experiment  8  -  45  sec  -  Ring  Finger  Grasping  Motion  with  Thumb  Restrained  -  Yellow  Stress  ball 
Experiment  9  -  60  sec  -  Ring  Finger  Grasping  Motion  with  Thumb  Restrained  -  Red  Stress  ball 

Experiments  1  and  2  were  conducted  to  give  insight  into  the  sEMG  generated  by  the  subjects  hand,  in  the  absence 
of  any  motion  or  force.  Also,  a  thumb  restrain  was  used  to  limit  the  influence  of  thumb  while  application  of  force  to 
the  stress  ball.  A  Force  Sensitive  Resistors  (FSR)  was  mounted  on  the  stress  ball  to  record  force  levels  for  different 
grasping  force  levels. 


Fig.  2.2.1.  Experimental  Setup  -  Location  of  sEMG  Sensors. 

Fig.  2.2.1  shows  the  variable  associated  with  the  various  sEMG  locations.  Fig.  2.2.2  shows  the  thumb -restrain  in  use 
during  various  contraction  experiments.  Fig.  2.2.3  also  shows  the  location  of  the  FSR.  Fig.  2.2.4  shows  the  sEMG 


capture  16  channel  Bagnoli  measuring  system. 
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Fig.  2.2.2.  Experimental  Setup  -  Variables  Associated  with  the  sEMG  Channels. 


Fig.  2.2.3.  Experimental  Setup  -  Location  of  Force  Sensitive  Resistor  (FSR)  and  Thumb  Restrain. 
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Fig.  2.2.4.  Experimental  Setup  -  Bagnoli  16  Channel  sEMG  Measuring  Device. 


The  most  commonly  used  Linear  Spatial  filters  for  isolating  the  motor  unit  action  potentials  (MUAPs)  are: 

1)  Longitudinal  Single  Differential  (LSD) 

2)  Transverse  Single  Differential  (TSD) 

3)  Longitudinal  Double  Differential  (LDD) 

4)  Transverse  Double  Differential  (TDD) 

5)  Normal  Double  Differential  (NDD) 

6)  Inverse  Binomial  (IB2)  and  the 

7)  Inverse  Rectangular  (IR)  Filter 

The  mask  of  these  filters  and  the  corresponding  resultant  equations  on  application  of  the  mask  to  the  grid  data 
obtained  from  the  sEMG  array  arrangement  are  given  below. 

CASE  1 


Longitudinal  Single  Differential  (LSD) 

EMG  Array  Information,  Spatial  Filter  Mask 


sEMGl 

sEMG5 

sEMG6 

0 

0 

0 

sEMG2 

sEMGl 

sEMG3 

-1 

1 

0 

sEMG  8 

sEMG4 

sEMG9 

0  0  0 


Result  Equation:  —  sEMG 2  +  sEMGl 


"0 

-1 

0“ 

"0 

0 

0" 

"0 

-1  0“ 

TSD= 

0 

1 

0 

LDD= 

-1 

2 

-1 

TDD  = 

0 

2  0 

_0 

0 

0_ 

_0 

0 

0_ 

_0 

—l  0_ 

5  5 


(4) 


(5) 


LSD  Equation  =  sEMGl-sEMG2;  TSD  Equation  =  sEMGl-sEMG5.  We  can  similarly  deduce  the  equations  for  the 
other  spatial  filters: 
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-1 
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"-1 

-2 

-f 

-1 

-1  -1 

NDD= 

-1 

4 

-1 

IB2= 

-2 

12 

-2 

IR= 

-1 

8  -1 

_0 

-1 

0_ 

-1 

-2 

-1 

-1 

-1  -1 
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In  contrast  to  the  varied  number  of  linear  spatial  filters  which  are  available  for  multichannel  surface  EMG 
signal  analysis  and  isolation  of  MUAP4s  there  is  a  dearth  of  nonlinear  spatial  filters.  Few  of  these  have  been 
reported  in  literature  [24].  The  nonlinear  spatial  filters  used  here  are  1)  1-D  Nonlinear  Transverse  Spatial  Filter,  2) 
1-D  Nonlinear  Fongitudinal  Spatial  Filter,  3)  2-D  Nonlinear  spatial  Filter  in  Two-  Orthogonal  Directions  and  4) 
Nonlinear  Spatial  Filter  in  All  Four  possible  Directions.  The  results  of  spatially  filtered  data  for  the  linear  case  and 
the  nonlinear  case  are  compared  based  on  the  -Kurtosis”  criteria.  Kurtosis  is  a  measure  of  whether  the  data  are 
peaked  or  flat  relative  to  a  normal  distribution.  That  is,  data  sets  with  high  kurtosis  tend  to  have  a  distinct  peak  near 
the  mean,  decline  rather  rapidly,  and  have  heavy  tails.  Data  sets  with  low  kurtosis  tend  to  have  a  flat  top  near  the 
mean  rather  than  a  sharp  peak.  A  uniform  distribution  would  be  the  extreme  case.  The  mathematical  expression  for 
kurtosis  is  given  in  equation  1 1 . 


Kurt  - 


£[x4]-3(£[x2])2 

E[x2f 


(7) 


The  Nonlinear  Spatial  Filters  use  the  Teager-Kaiser  Energy  (TKE)  Operator  [25],  [26].  This  operator  is  derived 
from  Teager's  experiments  in  1983.  This  technique  is  a  threshold  =energy4  based  approach  where  outliers  are  first 
detected  and  then  replaced  by  their  estimated  values.  Based  on  Newton4 s  law  of  motion,  a  nonlinear  quadratic 
operator  called  Teager-Kaiser  (TK)  operator  was  first  introduced  by  Teager  and  Kaiser  [25],  [26]  to  measure  the  real 
physical  energy  of  a  system.  This  nonlinear  operator  differs  from  the  common  way  to  calculate  the  energy  of  a 
discrete-time  signal  as  the  average  sum  of  its  squared  magnitudes.  The  energy  of  a  generating  system  of  a  simple 
oscillation  signal  was  computed  as  the  product  of  the  square  of  the  amplitude  and  the  frequency  of  the  signal.  It  was 
found  that  this  nonlinear  operator  exhibits  several  attractive  features  such  as  simplicity,  efficiency  and  ability  to 
track  instantaneously- varying  special  patterns.  Since  its  introduction,  several  applications  have  been  derived  for  one- 
dimensional  [27],  [28],  and  two  dimensional  signal  processing  [29]. 

Before  the  Teager-Kaiser  operator  was  introduced,  various  filtering  methods  were  proposed  both  linear  and 
nonlinear  alternatives.  Filtering  is  often  a  trade-off  between  different  features,  such  as  performance  in  Gaussian  and 
non-Gaussian  environments,  ability  to  adapt  in  case  of  non- stationary  signals,  edge  preservation  and  computational 
complexity.  Two  typical  methods  were  running  mean  and  median  filters,  which  exploit  a  sliding  fixed  length 
window.  Mean  filters  are  known  to  remove  the  additive  Gaussian  noise  very  well,  but  they  tend  to  soften  edges  and 
cannot  fully  cope  with  impulsive  noise.  Median  filters  on  the  other  hand  are  nonlinear,  which  involve  sorting  of  the 
data,  and  are  able  to  eliminate  impulses  at  the  expense  of  some  streaking  and  edge  jittering  [30].  De -noising  deals 
with  the  same  problem  of  estimating  the  underlying  signal  from  the  noisy  observations,  but  now  the  data  is  not 
needed  to  process  online  or  is  not  time -dependent  at  all.  De-noising  is  not  restricted  to  the  methods  acting  in  time 
domain,  the  signal  is  often  processed  in  some  transform  domain.  Wavelet  transform  has  recently  gained  research 
interest  in  several  fields  of  signal  processing,  using  thresholding  of  (orthogonal)  wavelet  coefficients  and  taking  the 
inverse  wavelet  transform  has  been  proposed  to  solve  de-noising  problem. 

The  nonlinear  spatial  filters  with  the  TKE  operator  incorporated  are  given  as  follows: 

a)  General  Form  of  Nonlinear  Spatial  Filter  using  the  Teager-Kaiser  (TKE)  operator 

vF[x(/2)]  =  x2  ( n )  x{n  +  l)x(n  - 1)  (8) 


b)  1-D  Nonlinear  Transverse  Spatial  Filter 

*¥d  m  \x(m ,  n)\  =  x2  (m,  n )  -  x(m  - 1,  n)x(m  + 1,  n ) 

c)  1-D  Nonlinear  Fongitudinal  Spatial  Filter 

x¥d  n  \x(m ,  n)\  =  x2  (m,  n )  -  x(m,  n  - 1  )x(m,  n  + 1) 

d)  Nonlinear  Spatial  Filter  in  Two  Orthogonal  Directions 


(9) 

(10) 
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'P/,2  OO,  «)]  =  \-X(m ’  «)]  +  'P,,.,  [*0>  «)] 

=2x2  (m, «)  -  x(m  - 1,  n)x(m  + 1, «)  -  x(m, «  -  l)x(m,  /?  + 1) 

e)  Nonlinear  Spatial  Filter  in  all  Four  Directions 


'F d  4  [x(m,  «)]  =  4x2  ( m,n)~  x(m  - 1,  ri)x(m  + 1,  n) 
-  x(m,  n  —  l)x(m,  n  + 1) 

-  x(m  - 1  ,n  + 1  )x(m  + 1,  n  - 1) 

-  x(m  - 1,  n  - 1  )x(m  + 1,  n  + 1) 


(11) 


Fig.  2.2.5  and  2.2.6  show  the  plots  of  the  raw  sEMG  signal  for  the  experiment  without  any  force  being  applied. 


Raw  sEMG  Data  for  Ring  Finger(at  MU)  Channel  1 
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Raw  sEMG  Data  for  Ring  Finger  Channel  3 


Raw  sEMG  Data  for  Ring  Finger  Channel  5 


Raw  sEMG  Data  for  Ring  Finger  Channel  2 


Raw  sEMG  Data  for  Ring  Finger  Channel  4 
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Raw  sEMG  Data  for  Ring  Finger  Channel  6 


Fig.  2.2.5.  Raw  sEMG  Data  Plots  Channels  1  to  6 
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Raw  sEMG  Data  for  Ring  Finger  Channel  7Raw  sEMG  Data  for  Ring  Finger  Channel  8 
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Fig.  2.2.6.  Raw  sEMG  Data  Plots  Channels  7  to  9 


The  sEMG  data  was  then  filtered  using  a  Chebyshev  Type  II  filter  with  a  pass-band  frequency  of  60-450  Hz.  The 
force  data  was  also  filtered  using  a  3rd  order  Low-pass  Butterworth  filter  with  cutoff  frequency  of  10  Hz.  The  same 
filtering  was  carried  out  for  all  the  experiments  conducted.  The  plots  of  the  filtered  force  and  sEMG  data  are  given 
in  Fig.  2.2.7.  Fig.  2.2.7  shows  the  baseline  sEMG  and  force  levels  generated  by  the  subject  where  no  contractions 
were  performed. 
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Fig.  2.2.7.  Filtered  sEMG  and  Force  Data  -  No  Force  Generation 


35 


Fig.  2.2.8  and  2.2.9  plots  the  raw  sEMG  of  all  the  channels  (1-9)  for  another  experiment  where  the  subject  was 
performing  the  action  of  squeezing  the  force  ball  with  a  force  sensor  attached  to  the  ball. 
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Raw  sEMG  Data  for  Ring  Finger  Channel  7  Raw  sEMG  Data  for  Ring  Finger  Channel  8 


Raw  sEMG  Data  for  Ring  Finger  Channel  9 
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Fig.  2.2.9.  Raw  sEMG  Data  Plots  Channels  7  to  9 

Fig.  2.2.10  shows  the  sEMG  and  the  force  signal  generated  by  the  subject  where  the  subject  performed  random 
variation  of  force  by  squeezing  the  stress  ball. 
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Filtered  Force  and  Filetred  sEMG  Data  Channel  1 


Fig.  2.2.11  (a),  (b),  (c)  and  (d)  show  the  results  of  the  Non-linear  Spatially  Filtered  sEMG  data  for  the  multi-sensor 
array  setup  shown  in  Fig.  2.2.1. 


1  D  Spatial  Filter  Transverse  to  the  Muscle  Fiber  Direction 


(a) 


1  D  Spatial  Filter  in  TWO  Orthogonal  (TO)  Directions  to  the  Muscle  Fiber 


(b) 
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Fig.  2.2.1 1.  Spatially  Filtered  sEMG  a)  1-D  Nonlinear  Transverse  Spatial  Filter,  b)  1-D  Nonlinear  Longitudinal 
Spatial  Filter,  c)  Nonlinear  Spatial  Filter  in  Two  Orthogonal  Directions,  d)  Nonlinear  Spatial  Filter  in  All  Four 

Directions 
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Fig.  2.2.12  shows  the  results  of  the  various  Linear  Spatially  Filtered  sEMG  data  for  the  multi-sensor  array  setup 
shown  in  Fig.  2.2.1. 
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Longitudinal  Single-Differential  Filter  (LSD) 
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Fig.  2.2.12.  Spatially  Filtered  sEMG  a)  Longitudinal  Single  Differential  (LSD),  b)  Longitudinal  Double  Differential 
(LDD),  c)  Normal  Double  Differential  (NDD),  d)  Inverse  Rectangular  (IR)  Filter,  e)  Transverse  Single  Differential 
(TSD),  f)  Transverse  Double  Differential  (TDD),  g)  Inverse  Binomial  (IB2) 


The  sEMG  signal  was  subjected  to  various  spatial  filters  as  mentioned  in  the  previous  section.  In  addition  to  these 
1 1  spatial  filters,  the  sEMG  data  was  also  filtered  using  4  other  filters  -  Bessel,  Butterworth,  Chebyshev  Type  I  and 
Chebyshev  Type  II  filters.  The  filter  characteristics  of  these  4  filters  were  in  accordance  to  the  ISEK  standards.  The 
relation  of  sEMG-Force  was  modeled  using  Non-Linear  Hammerstein- Wiener  models.  This  modeling  method  has 
been  proven  to  work  in  the  past,  in  assuming  a  black-box  model  structure  for  the  system.  In  order  to  model  the 
sEMG-Force  relation,  the  filtered  data  from  the  various  experiments  was  split  into  various  four  (4)  time  windows. 
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phixmnTDD  phixmnTSD 


Equation 

(12)  describes  the 

general 

Hammerstein- W  iener 

model 

structure: 

Bn{q) 

m  =  y(t)  =  h(x(t)), 

FhM) 

(12) 

where,  w(t)  and  b(t)  are  internal  variables,  w(t)  has  the  same  dimensions  as  u(t)  -  input,  and  x(t)  has  the  same 
dimensions  as  y(t)  -  output.  g()  and  h()  are  the  input  and  output  non-linearity  functions  respectively.  B(q)  and 
F(q)  are  regression  polynomials.  The  model  fit  values  are  computed  using  Equation  (13) 
as  follows: 

fit  =  (13) 

\\y— y\\ 

where,  y  is  the  estimated  output  by  the  model. 

The  time  windows  used  for  estimation  and  validation  of  the  models  were  called  ^ze"and  jv" respectively,  je” 
contained  8000  sample  points  and  jv  "contained  data  points  shifted  by  2000  sample  points.  For  example,  if  =ze"was 
a  time  window  between  2-6  seconds  i.e.  samples  4000-12000,  then  ^vffwas  between  3-7  seconds  i.e.  6000-14000 
samples.  Thus  the  Hammer  stein- Wiener  method  uses  je”  to  estimate  the  model  structure  and  based  on  this 
information  predicts  the  next  2000  sample  points.  The  data  was  filtered  using  the  various  filters  mentioned  in  the 
previous  sections.  We  would  like  to  stress  once  again  the  point  that  the  force  was  varied  randomly  and  the  subject 
was  in  no  way  trying  to  achieve  maximum  voluntary  contractions  during  each  cycle.  A  cycle  is  defined  as  the 
subject  starting  without  any  force  on  the  stress  ball,  squeezing  it  (to  any  force  level)  and  then  going  back  to  no  force. 
The  subject  has  to  keep  the  finger  in  contact  with  the  force  ball  throughout  this  cycle. 

A  plot  of  the  variations  in  force  achieved  for  3  experiments  is  shown  in  Fig.  2.2.13.  The  Matlab®  code  for  the 
Hammer  stein- Wiener  model  is:  nlhw(ze,  [ na  n ^  nk\ ,  ....,  ...).  The  modeling  was  carried  out  by  varying  na  -  the 

number  of  past  output  terms  used  to  predict  the  current  output,  n ^  -  the  number  of  past  input  terms  used  to  predict 

the  current  output  and  n^-  the  delay  from  input  and  output  in  terms  of  the  number  of  samples  for  the  various 
Hammerstein- Wiener  models. 


models  estimated  were  15  (filter  types)  x  4  (time  windows)  x  42  models  per  time  window  x  4  experiments  *  10,080 
models. 

(Refer  for  additional  details  Publication  — Evaluation  of  Filtering  Techniques  Applied  to  Surface  EMG  Data  and 
Comparison  Based  on  Hammerstein- Wiener  Models”)  [PI] 

Table  2.2.2  and  2.2.3  show  the  results  of  modeling  for  the  time  window  0-9  sec.  42  models  were  estimated  by 
varying  na  and  n b  only  35  of  those  are  shown  here  on  account  of  page  size  limitation.  The  Hammerstein- Wiener 

models  worked  very  well  in  capturing  the  dynamics  of  the  force  levels  for  the  various  experiments  conducted.  This 
method  of  modeling  could  help  in  improving  the  control  over  the  motors  used  in  prosthetic  devices  to  mimic  the 
actual  changes  force  levels  in  a  real  hand.  This  method  also  performed  very  well  in  the  scenarios  where  the  subject 
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did  fatigue  but  the  affects  were  successfully  modeled  by  the  Hammer stein-Wiener  models.  The  nonlinear  and  linear 
spatial  filters  (TDD,  NDD  and  NLT,  NLTO  and  NLAFD)  did  outperform  the  other  filtering  methods  used  especially 
for  the  later  time  windows.  The  only  other  filter  which  had  a  comparable  performance  to  the  spatial  filters  was  the 
Bessel  filter.  Further  investigation  into  reducing  the  wide  range  of  the  fit  values  obtained  needs  to  be  performed. 

One  of  the  possible  methods  to  pursue  would  be  to  use  Genetic  Algorithm  to  optimize  the  model  parameters  na  and 
rifr  and  also  the  number  of  iterations  used  for  the  modeling  of  sEMG-force  levels.  One  of  the  possible  reasons  for 
poor  fit  values  could  also  be  attributed  to  the  model  trying  to  over-fit  the  data  sets. 

We  pursued  optimization  of  the  filter  mask  using  GA  and  the  details  of  this  investigation  are  presented  in  detail  in 
the  paper  titled  -Optimized  Spatial  Filter  Mask  Using  Genetic  Algorithm  And  sEMG-Force  Modeling  Using  System 
Identification”.  [P2] 


TABLE  2.2.2 

EXAMPLE  OF  MODEL  FIT  VALUES  FOR  TIME  WINDOW  0-9  SEC  -  VARIED  NA  AND  NB 


Model 

Number 

na 

nb 

nk 

Bessel 

Butter-worth 

Cheby  I 

Cheby  II 

NLT 

NLL 

NLTO 

ml 

2 

3 

1 

40.67 

10.37 

34.65 

13.09 

34.17 

17.52 

60.13 

m2 

2 

4 

1 

53.19 

41.72 

31.75 

30.02 

31.25 

12.08 

10.54 

m3 

2 

5 

1 

51.32 

29.32 

13.5 

1.644 

35.51 

37.41 

32.78 

m4 

2 

6 

1 

56.36 

2.764 

12.19 

36.28 

34.07 

13.87 

33.57 

m5 

2 

7 

1 

54.23 

11.82 

31.93 

35.53 

39.7 

28.23 

28.91 

m6 

2 

8 

1 

53.2 

11.05 

12.01 

36.59 

19.71 

26.51 

26.17 

ml 

2 

9 

1 

46.88 

32.3 

32.7 

-0.297 

33.8 

15.8 

35 

m8 

3 

3 

1 

52.05 

31.56 

30.21 

36.3 

34.64 

19.16 

17.8 

m9 

3 

4 

1 

48.66 

6.734 

19.66 

39 

36.45 

19.74 

33.04 

mlO 

3 

5 

1 

13.79 

34.23 

32.11 

36.06 

17 

17.92 

16.44 

mil 

3 

6 

1 

51.9 

42.87 

15.19 

36.39 

33.75 

17.84 

19.51 

ml  2 

3 

7 

1 

50.05 

18.56 

2.805 

36.66 

34.7 

29.34 

34.02 

ml  3 

3 

8 

1 

51.74 

26.33 

33.82 

43.69 

28.41 

27.05 

32.79 

ml4 

3 

9 

1 

33.48 

34.44 

25.83 

37 

32.56 

27 

18.55 

ml  5 

4 

3 

1 

48.02 

18.53 

12.38 

39.07 

15.27 

26.56 

15.54 

ml  6 

4 

4 

1 

52.36 

35.9 

15.09 

3.257 

14.95 

26.84 

32.16 

ml7 

4 

5 

1 

12.1 

40.11 

14.74 

30.87 

27.77 

17.76 

40.91 

ml8 

4 

6 

1 

54.11 

1.436 

14.84 

38.77 

27.45 

30.69 

16.55 

ml9 

4 

7 

1 

50.38 

30.62 

27.6 

35.76 

46.37 

23.5 

27.66 

m20 

4 

8 

1 

44.06 

-0.4972 

9.644 

36.71 

36.58 

26.63 

11.47 

m21 

4 

9 

1 

50.44 

1.269 

-0.0622 

37.15 

36.07 

40.85 

34.55 

m22 

5 

3 

1 

48.26 

34.89 

13.48 

17.98 

26.48 

18.41 

58.38 

m2  3 

5 

4 

1 

51.51 

5.417 

13.53 

38.45 

39.21 

25.89 

46.88 

m24 

5 

5 

1 

52.61 

43.02 

12.5 

36.78 

33.26 

17.92 

25.49 

m2  5 

5 

6 

1 

47.45 

10.01 

12.22 

36.45 

34.25 

27.74 

27.41 

m2  6 

5 

7 

1 

52.25 

30.03 

14.07 

36.29 

33.95 

30.69 

27.12 

m2  7 

5 

8 

1 

52.22 

31.85 

12.61 

37.87 

34 

28.33 

26.55 

m2  8 

5 

9 

1 

55.21 

33.96 

12.32 

31.98 

57.96 

26.95 

25.52 

m2  9 

6 

3 

1 

50.12 

-0.3022 

-0.1301 

29.63 

15.48 

30.51 

16.15 
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m30 

6 

4 

1 

43.39 

21.07 

30.2 

2.117 

34.32 

25.46 

37.36 

m31 

6 

5 

1 

49.26 

1.169 

19.23 

10.32 

36.3 

31.73 

16.04 

m3  2 

6 

6 

1 

53.1 

15.03 

14.63 

9.181 

26.98 

14.11 

26.16 

m3  3 

6 

7 

1 

52.9 

0.9796 

15.35 

0.6897 

46.02 

33.64 

34.97 

m3  4 

6 

8  1 

51.92 

5.604 

11.16 

1.212 

27.86 

46.24 

16 

m3  5 

6 

9 

1 

51.35 

21.36 

19.06 

39.84 

41.41 

27.68 

20.68 

TABLE  2.2.3 


EXAMPLE  OF  MODEL  FIT  VALUES  FOR  TIME  WINDOW  0-9  SEC  -  VARIED  NA  AND  NB 


Model 

Number 

na 

nb 

nk 

NLAFD 

LSD 

TSD 

LDD 

TDD 

NDD 

IB2 

IR 

ml 

2 

3 

1 

17.21 

41.39 

-9.685 

31.06 

39.53 

45.55 

33.79 

35.38 

m2 

2 

4 

1 

16.75 

17.4 

43.19 

31.86 

39.36 

36.84 

16.88 

23.89 

m3 

2 

5 

1 

17.99 

35.94 

2.247 

23.54 

30.09 

39.87 

36.17 

38.16 

m4 

2 

6 

1 

20.51 

41.99 

43.46 

3.266 

41.28 

10.83 

36.41 

36.7 

m5 

2 

7 

1 

34.41 

40.55 

31.28 

31.58 

44.65 

2.593 

32.26 

36.34 

m6 

2 

8 

1 

12.24 

4.259 

20.08 

37.84 

21.89 

8.036 

36.22 

36.57 

ml 

2 

9 

1 

31.1 

-0.1887 

28.71 

31.42 

35.78 

38.11 

35.54 

35.82 

m8 

3 

3 

1 

18.97 

40.04 

12.33 

36.59 

36.17 

4.478 

41.09 

35.81 

m9 

3 

4 

1 

25.54 

40.72 

36.57 

20.74 

31.53 

29.76 

37.43 

38.76 

mlO 

3 

5 

1 

22.03 

11.53 

30.24 

39.87 

39.83 

39.25 

35.15 

37.9 

mil 

3 

6 

1 

18.37 

35.41 

44.24 

38.79 

40.76 

39.01 

35.66 

38.17 

ml2 

3 

7 

1 

17.87 

41.67 

42.69 

35.54 

39.93 

36.81 

31.31 

35.6 

ml3 

3 

8 

1 

23.09 

0.2404 

2.988 

37 

39.6 

37.4 

36.18 

15.24 

ml4 

3 

9 

1 

5.519 

39.42 

44.21 

37.83 

26.8 

41.36 

37.53 

34.42 

ml5 

4 

3 

1 

21.62 

41.48 

15.21 

34.4 

43.01 

39.58 

0.8962 

30.42 

ml  6 

4 

4 

1 

35.08 

37.8 

29.57 

37.76 

40.12 

-0.3683 

36.65 

36.9 

ml  7 

4 

5 

1 

32.36 

40.53 

20.3 

23.31 

17.97 

32.97 

37.22 

34.48 

ml8 

4 

6 

1 

31.55 

41.86 

33.33 

33.27 

22.13 

30.85 

38.48 

39.23 

ml9 

4 

7 

1 

17.66 

41.51 

42.28 

36.6 

41 

41.16 

38.11 

35.41 

m20 

4 

8 

1 

20.88 

42.04 

29.77 

35.3 

40.65 

18.03 

37.3 

37.03 

m21 

4 

9 

1 

18.93 

41.33 

37.4 

27.97 

36.73 

6.661 

37.54 

36.21 

m22 

5 

3 

1 

18 

14.08 

41.46 

2.748 

42.14 

41.36 

33.82 

34.69 

m2  3 

5 

4 

1 

25.55 

7.815 

42.19 

30.54 

39.49 

36.47 

37.93 

35.36 

m24 

5 

5 

1 

17.81 

38.31 

30.71 

39.16 

41.95 

36.29 

34.53 

16.75 

m2  5 

5 

6 

1 

17.72 

16.64 

31.41 

37.13 

41.81 

37.15 

36.03 

36.3 

m2  6 

5 

7 

1 

20.48 

33.97 

37.15 

38.43 

34.96 

34.03 

29.84 

35.49 

mil 

5 

8 

1 

16.78 

34.58 

29.29 

18.8 

41.45 

39.14 

35.47 

34.89 

m2  8 

5 

9 

1 

15.89 

35.33 

36.44 

37.96 

42.36 

40.24 

21.59 

35.36 

m2  9 

6 

3 

1 

18.76 

42.76 

30.01 

38.74 

39.93 

37.22 

40.46 

-0.1423 

m30 

6 

4 

1 

22.08 

38.44 

29.07 

38.81 

39.93 

29.86 

33.32 

33.93 
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m31 

6 

5 

1 

17.71 

42.58 

29.1 

39.44 

38.74 

36.42 

36.17 

38.93 

m3  2 

6 

6 

1 

16.67 

38.6 

41.18 

19.42 

40.12 

40.96 

35.99 

35.34 

m3  3 

6 

7 

1 

20.38 

42.05 

31.74 

10.04 

39.55 

43.21 

35.88 

31.28 

m3  4 

6 

8  1 

17.05 

3.922 

25.66 

28.73 

39.85 

39.5 

31.34 

13 

m3  5 

6 

9 

1 

17.24 

-0.4172 

42.73 

15.23 

39.03 

36.42 

21.46 

34.38 

Some  of  the  models  for  this  time  window  were  selected  and  then  recomputed  using  GA  to  optimize  the  filter 
mask  with  the  fitness  function  being  the  fit  value  of  an  identified  model  achieved  for  a  given  model  order.  Table 
2.2.4  shows  an  example  of  the  fits  that  were  obtained  on  varying  the  parameters  of  the  Hammer  stein -Wiener  model. 
The  highlighted  models  (and  a  few  other  models)  were  selected  randomly  to  be  optimized  using  GA.  We  had  two 
scenarios  under  GA  -1)  GA  Constrained  and  2)  GA  -  Unconstrained.  The  first  scenario  GA  constrained  optimized 
only  the  mask  entry  a22  (location  of  the  sEMG  sensor  on  the  motor  unit).  The  other  entries  of  the  filter  mask  were 
then  computed  from  this  optimized  value.  In  the  second  scenario,  GA  unconstrained  we  let  GA  optimize  all  the 
entries  for  various  masks.  The  GA  parameters  for  optimization  were  as  follows:  number  of  iterations:  50;  initial 
population  size  Generation  0:  96;  population  size  Generation  1:  48;  Number  of  Chromosomes  kept  for  mating:  24; 
and  mutation  rate  was  set  to  4%. 


TABLE  2.2.4 

EXAMPLE  OF  SYSTEM  IDENTIFICATION  RESULTS  USING  FILTER  MASK  FROM  LITERATURE  -  HIGHLIGHTED  MODELS 

OPTIMIZED 


Model 

Number 

na 

nb 

nk 

LDD 

LSD 

TDD 

TSD 

NDD 

IB2 

IR 

ml 

2 

3 

1 

31.06 

41.39 

39.53 

-9.685 

45.55 

33.79 

35.38 

m2 

2 

4 

1 

31.86 

17.4 

39.36 

43.19 

36.84 

16.88 

23.89 

m3 

2 

5 

1 

23.54 

35.94 

30.09 

2.247 

39.71 

fssm 

38.16 

m4 

2 

6 

1 

3.266 

41.99 

41.28 

43.46 

10.83 

36.41 

36.7 

m5 

2 

7 

1 

31.58 

40.55 

44.65 

31.28 

2.593 

32.26 

36.34 

m6 

2 

8 

1 

37.84 

4.259 

21.89 

20.08 

8.036 

36.22 

36.57 

m7 

2 

9 

1 

31.42 

-0.1887 

35.78 

28.71 

38.11 

35.54 

35.82 

m8 

3 

3 

1 

36.59 

40.04 

36.17 

12.33 

4.478 

41.09 

35.81 

m9 

3 

4 

1 

20.74 

40.72 

31.53 

36.57 

29.76 

37.43 

38.76 

mlO 

3 

5 

1 

39.87 

11.53 

39.83 

30.24 

39.25 

35.15 

37.9 

m40 

7 

7 

1 

34.91 

10.18 

40.79 

35.21 

35.24 

33.42 

37.92 

m41 

7 

8 

1 

-0.1127 

36.28 

39.68 

40.26 

20.27 

35.88 

24.32 

m42 

7 

9 

1 

36.55 

-0.412 

25.24 

31.31 

35.4 

34.26 

39.82 
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TABLE  2.2.5  (A) 

RESULTS  OF  CONSTRAINED  AND  UNCONSTRAINED  GA  OF  HIGHLIGHTED  MODELS  FROM  TABLE  1 


Longitudinal  Double  Differential  (LDD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

31.06 

42.8291 

61.3475 

m6 

37.84 

53.3134 

33.724 

ml2 

35.54 

44.2513 

51.7942 

m3! 

39.44 

47.0034 

60.6489 

Transverse  Double  Differential  (TDD) 


Fit  % 


From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

39.53 

41.5297 

59.8055 

m5 

44.65 

49.4667 

68.4191 

mlO 

39.83 

41.307 

63.8523 

ml5 

43.01 

43.3466 

56.5113 

Longitudinal  Single  Differential  (LSD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

41.39 

48.9135 

59.8055 

m4 

41.99 

41.5731 

68.4191 

m8 

40.04 

40.5924 

63.8523 

m9 

40.72 

45.2322 

56.5113 

Transverse 

Single  Differential  (TSD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

m2 

43.19 

44.0017 

65.9038 

m4 

43.46 

43.0529 

58.2755 

mil 

44.24 

39.7867 

44.8431 

m31 

29.1 

47.6688 
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TABLE  2.2.5  (B) 

RESULTS  OF  CONSTRAINED  AND  UNCONSTRAINED  GA  OF  HIGHLIGHTED  MODELS  FROM  TABLE  3 


Normal  Double  Differential  (NDD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

45.55 

53.1512 

57.9496 

m3 

39.87 

47.1084 

60.7374 

ml 

38.11 

51.8065 

60.5535 

mil 

39.01 

56.1193 

Inverse  Binomial  2  (IB2) 


Fit  % 

From 

GA 

GA 

Literature  Constrained 

Unconstrained 

ml 

33.79 

38.6076 

58.2365 

m3 

36.17 

38.9074 

58.1653 

m4 

36.41 

38.9957 

55.5179 

m8 

41.09 

47.2971 

55.0537 

From  the  results  in  Table  2.2.5  (A)  &  (B)  we  can  see  that  the  optimization  of  the  filter  mask  using  GA  worked  in 
almost  all  the  cases  chosen.  GA  without  constraints  performed  significantly  better,  in  most  cases,  than  the  filter 
masks  reported  in  literature  and  also  the  mask  which  we  computed  using  GA,  which  only  optimized  the  entry  (a22) 
i.e.  the  weight  associated  with  the  sEMG  signal  at  the  motor  unit.  This  restriction  on  GA  would  leave  the  filter  mask 
symmetrical.  But  looking  at  the  results  of  the  GA,  we  can  conclude  that  the  filter  mask  need  not  always  be 
symmetrical  for  analysis  of  sEMG,  especially  for  data  recorded  using  an  array.  Examples  of  the  filter  mask  that  we 
obtained  for  TDD,  model  m5  and  NDD,  model  3  spatial  filters  are  given  below: 


"0 

-41.5561 

0" 

0 

-11.5790 

0 

TDD  = 

0 

54.7329 

0 

NDD  = 

-46.7173 

36.9276 

-11.8061 

0 

-22 

0 

0 

-70.0491 

0 

The  linear  spatial  filter  masks  reported  in  literature  had  poor  kurtosis  improvements  which  we  optimized  using  GA, 
these  were  compared  based  on  the  model  fit  values  achieved  using  Hammerstein- Wiener  models.  The  fit  values  did 
improve  significantly  in  the  two  GA  scenarios  -  GA  with  and  without  constraints.  The  GA  without  constraints 
performed  better  than  the  GA  with  constrains,  which  brings  into  focus  the  possibility  that  the  sEMG  signal 
distribution  over  the  entire  grid  cannot  be  assumed  to  be  symmetrically  distributed  and  that  the  weights  associated 
with  the  sEMG  signal  at  various  locations  need  to  be  modified  depending  on  probably  the  subject  and  also  based  on 
the  experimental  design.  This  is  in  contrast  to  the  reported  filter  mask  in  the  literature,  which  are  all  symmetric. 
Almost  all  the  filter  masks  optimized  resulted  in  a  significant  improvement  over  the  masks  reported  in  literature. 

2.2.2  Signal  Pre-processing: 

An  experimental  set-up  is  designed  with  a  5”  circular  FSR  from  Interlink™  Electronics  and  a  stress  ball.  The 
experimental  set-up  is  shown  in  Fig.  2.2.2(a).  Using  this  experimental  set-up  both  sEMG  and  muscle  force  signals 
were  acquired  simultaneously  using  Lab  VIEW™  at  a  sampling  rate  of  2000  Hz.  The  sEMG  data  capturing  was 
aided  by  a  DELSYS®  Bagnoli-16  EMG  system  with  DE-2.1  differential  EMG  sensors.  One  sEMG  sensor  was 
placed  on  the  motor  point  of  the  ring  finger  and  two  adjacent  to  the  motor  point  of  a  healthy  subject.  Prior  to  placing 
the  sEMG  sensors,  the  skin  surface  of  the  subject  was  prepared  according  to  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK)  protocols. 
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Fig.  2.2.2(a).  Experimental  Set-Up. 


Pre-processing  of  the  signals  captured  was  done  according  to  previous  research  based  on  [31],  where  the 
Bayesian  based  filtering  method  yields  the  most  suitable  sEMG  signals.  The  nonlinear  filter  significantly  reduces 
noise  and  extracts  a  signal  that  best  describes  EMG  signals  and  may  permit  effective  use  in  prosthetic  control.  An 
instantaneous  conditional  probability  density  P(EMG  \  x )  provides  the  resulting  EMG  for  the  latent  driving  signal  x 
[31].  The  model  for  the  conditional  probability  of  the  rectified  EMG  signal  &mg  =  \EMG\  is  used  in  this  current 
estimation  algorithm.  EMG  signals  are  usually  described  as  amplitude -modulated  zero  mean  Gaussian  noise 
sequence  [32].  For  the  rectified  EMG  signal,  the  -Half-Gaussian  measurement  model”  in  [31]  is  given  by  Equation 
(2.2.2. 1). 

P(&mg  |  x)  =  2  *  exp(—  A2/(2  **xA2))/  /(2  *  te  *;rA2)JA(l/2}.  (2.2.2. 1) 

The  EMG  signal  is  modeled  for  the  conditional  probability  of  the  rectified  EMG  signal  as  a  filtered  random 
process  with  random  rate.  The  likelihood  function  for  the  rate  evolves  in  time  according  to  a  Fokker-Planck  partial 
differential  equation  [31].  The  discrete  time  Fokker-Planck  Equation  is  given  by  Equation  (2. 2.2. 2). 

p  (-T,  t-')  —  —  T-  (1  -  2  *  a)  *  p{xf  £  -  1)  -F  a  *  p(x  +  t  -  1)  -F  ft  -F  (1  -  f¥)  #  p{x,  t  - 

i>- 

(2. .2. 2.2) 

Here,  a  and  8  are  two  free  parameters,  a  is  the  expected  rate  of  gradual  drift  in  the  signal,  and  is  the  expected 
rate  of  sudden  shifts  in  the  signal.  The  unknown  driving  signal  is  discretized  into  bins  of  width  f.  These  two  free 
parameters  of  the  non-linear  Half-Gaussian  filter  model  are  optimized  for  the  acquired  EMG  data  using  elitism 
based  Genetic  Algorithm  (GA).  GA  belongs  to  a  class  of  optimization  algorithms  that  are  based  on  observing  nature 
and  its  corresponding  processes  to  imitate  solving  complex  problems,  most  often  optimization  or  estimation 
problems,  see  [33-35].  A  Chebyshev  type  II  low  pass  filter  with  a  550  Hz  pass  frequency  is  used  to  filter  the  force 
signal.  Fig.  2.2.2(b)  depicts  the  raw  and  Chebyshev  type-II  low  pass  filtered  force  signals. 


(a)  Raw  Skeletal  Muscle  Force  Signal 


Fig.  2.2.2(b).  (a)  Raw  and  (b)  Chebyshev  Type-II  Filtered  Skeletal  Muscle  Force  Signals. 
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Rectified  EMG  for  Motor  Point  Sensor 


Fig.  2.2.2(c).  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Motor  Point  Sensor. 


Fig.  2.2.2(d).  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Ring  1  Sensor. 


Rectified  EMG  for  Ring  2  Sensor 


Half  Gaussian  Filtered  EMG  for  Ring  2  Sensor 


Time  (60.35  Seconds) 


x  lO4 


Fig.  2.2.2(e).  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Ring  2  Sensor. 


Fig.  2.2.2(c),  (d)  and  (e)  show  the  rectified  EMG  and  Half-Gaussian  Filtered  EMG  signal  for  three  sensors.  After 
this,  experiments  with  an  array  of  nine  sensors  were  conducted.  The  experimental  set-up  for  the  ring  finger 
experiment  is  shown  in  Fig.  2.2.2(f)  and  2.2.2(g). 
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Fig.  2.2.2(f).  Experimental  Set-Up  for  Ring  Finger  with  Nine  sEMG  Sensors  and  Red  Stress  Ball. 

In  this  case,  there  is  an  array  of  nine  sensors;  where  the  motor  point  sensor  is  at  the  center  of  the  array.  With  this 
experimental  set-up,  nine  sets  of  experiments  with  different  time  duration  and  with  two  different  stress  balls  of 
different  stiffness  and  with  and  without  of  thumb  restrictor  were  conducted.  Fig.  2.2.2(h)  shows  the  corresponding 
30  seconds  time -amplitude  plot  for  the  ring  finger  associated  EMG  signal  and  above-mentioned  experimental  set-up. 


Fig.  2.2.2(g).  Experimental  Set-Up  for  Ring  Finger  with  Nine  sEMG  Sensors  and  Yellow  Stress  Ball. 


1  2  3  4  5  6 

Time  x  io4 


Fig.  2.2.2(h).  Ring  Finger  Motor  Point  sEMG  Signal  for  Nine-Sensor  Array. 

This  sEMG  signal  is  filtered  using  nonlinear  spatial  filtering  for  multichannel  surface  EMG.  As  given  in  [36], 
this  nonlinear  spatial  filter  is  derived  from  -Nonlinear  Teager-Kaiser  Energy  (TKE)  Operator.”  In  [36],  the 
schematic  description  of  the  multiple  channel  surface  EMG  recording  is  given  in  Fig.  2.2.2(i).  Here  the  array  is  of 
nine  sensors  covers  the  four  directional  spatial  distribution  of  the  sEMG  signal. 
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Fig  2.2.2(i).  Schematic  Description  of  the  Multiple  Channel  Surface  EMG  Recording  [36]. 


As  given  in  [36],  the  TKE  operator  in  time  domain  on  sEMG  signal  is  defined  by  Equation  (2. 2. 2. 3). 

^[xGO]  =  x2(n)  -  x(n  El]3*  x(n  -  l).  (2.2.2. 3) 

Here,  W  is  TKE  operator  and  x(n)  is  time  domain  sEMG  signal.  Based  on  the  TKE  operator,  the  one-dimensional 
nonlinear  spatial  filters  are  given  by  Equation  (2. 2. 2.4)  and  (2. 2. 2. 5).  Two-dimensional  and  four-dimensional 
nonlinear  spatial  filters  are  given  by  Equation  (2. 2. 2. 6)  and  (2. 2. 2. 7)  respectively. 

=  _x2(m, n)  -x(m  -  l,n)  (2.2.2.4) 

W^iTl |>(m. ft) ]  =  x2(m,n)  -  x( mfn  -  l)  u(m,n+  1).  (2.2.2. 5) 

2[x(mJn)]  =  2  *  x3(m,n)  —  x(m  —  l,n)  *  E  l,n)  —  xtm.n  —  l)  *  ;c(m,n  E  1).  (2. 2. 2. 6) 

^  4[x(.mf  n)]  =  4  *  x2(m, n)  —  x{m  —  1, ?i]  *  x{m  ELw)  -  x{mt n  —  l)  *  x{m, n  E  l)  —  x[m  -  Ln  E  l)  * 
x(m  E  l,n  —  l)  —  x(m  —  l,n  —  1)  *  EfnEl) 

(2.2.2. 7) 

Fig.  2.2.2(j)  and  2.2.2(k)  shows  the  one-dimensional  nonlinear  spatially  filtered  data  plots  for  ring  finger.  Fig. 


2. 2. 2(1)  and  2.2.2(m)  shows  the  two  and  four-dimensional  nonlinear  spatially  filtered  data  plots  for  ring  finger 
signals  respectively. 


l-Directional  Spatially  Filtered  Data  -  Along  Muscle  Fiber 


Time  x  io4 

Fig.  2.2.2(j).  One-Directional  -  Along  the  Muscle  Fiber  -  Nonlinear  Spatially  Filtered  Data  for  Ring  Finger 

sEMG  Signals. 
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1 -Directional  Spatially  Filtered  Data  -  Orthogonal  to  Muscle  Fiber 


1  2  3  4  5  6 
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Fig.  2.2.2(k).  One-Directional  -  Orthogonal  to  Muscle  Fiber  -  Nonlinear  Spatially  Filtered  Data  for  Ring  Finger 

sEMG  Signals. 


1  2  3  4  5  6 
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Fig.  2. 2. 2(1).  Two-Dimensional  Nonlinear  Spatially  Filtered  Data  for  Ring  Finger  sEMG  Signals. 


4-Directional  Spatially  Filtered  Data  -  An  Array  of  Nine  Sensors 


Fig.  2.2.2(m).  Four-Dimensional  Nonlinear  Spatially  Filtered  Data  for  Ring  Finger  sEMG  Signals. 

These  nonlinear  spatially  filtered  data  is  not  yet  modeled  using  SI.  In  the  future  work,  we  will  model  these 
nonlinear  spatially  filtered  data  for  different  fingers  and  different  time  durations. 

We  designed  a  new  experimental  set-up  to  simultaneously  acquire  the  sEMG  from  the  motor  point  of  the  index 
finger  and  the  joint  angle  of  the  PIP  joint  of  the  index  finger.  Nine  DE-3.1  sEMG  sensors  of  the  DELSYS® 
Bagnoli-16  EMG  system  are  placed  in  a  three -by-three  array  [36];  the  arrangement  covers  the  four  directional 
spatial  distributions  of  the  sEMG  signal.  The  middle  three  sEMG  sensors  were  attached  directly  on  the  skin  surface 
above  the  motor  point  of  the  index  finger  of  a  healthy  subject.  Using  a  lOk-Ohm  wheel  potentiometer,  an  angle 
measurement  device  is  designed  to  measure  the  joint  angles  of  the  PIP  joint  of  the  index  finger  of  a  healthy  subject. 
Appropriate  sEMG  electrode  attachment  point  for  the  motor  point  of  the  index  finger  was  identified  using  a  wet 
probe  muscle  stimulator  at  the  FDS  (RICH -MAR,  HV  1000). 
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Fig.  2.2.2(n):  Experimental  Set-Up  for  Angle  Measurement. 

Prior  to  placing  the  sEMG  sensors,  the  skin  surface  of  the  subject  was  prepared  according  to  International 
Society  of  Electrophysiology  and  Kinesiology  (ISEK)  protocols.  An  Interlink™  Electronics  FSR  0.5”  circular  force 
sensor  on  a  stress  ball  is  used  to  press  with  the  movement  to  introduce  some  resistance  to  the  movement  of  the  PIP 
joint  of  the  index  finger.  Experimental  set-up  is  shown  in  Fig.  2.2.2(n)  where  nine  sensors  are  shown  on  a  healthy 
subject  forearm,  with  an  angle  measurement  device  on  the  PIP  joint  of  the  index  finger  and  having  a  stress  ball  for 
force  measurements.  The  various  signals  are  sampled  at  a  rate  of  2000  Hz  using  Lab  VIEW™  in  conjunction  with 
DELSYS®  Bagnoli-16  EMG  and  NI  ELVIS™.  With  this  experimental  set-up  we  conducted  several  experiments  of 
30  seconds,  45  seconds  and  60  seconds  durations. 

To  investigate  the  change  in  sEMG  signal  as  a  result  of  muscle  fatigue  we  conducted  two  different  sets  of  fatigue 
inducing  experiments.  One  experiment  using  dynamic  force  variations  and  another  with  50  pounds  of  static  force 
was  conducted.  For  the  dynamic  force  variation  we  used  an  Interlink™  Electronics  FSR  0.5”  circular  force  sensor 
on  a  stress  ball  and  for  the  static  force  experiment  we  used  a  cable  tensionmeter  (T5166)  by  pacific  Scientific 
Company.4  Fig.  2.2.2(n)  and  (o)  shows  the  experimental  set-up  for  dynamic  force  variations  and  static  force  level. 
For  the  dynamic  force  experiment  we  restricted  the  thumb  movement  using  a  thumb  splint.  For  the  static  force 
experiment  we  held  the  force  of  the  dynamometer  at  50  pounds  and  tried  to  maintain  this  force  level  to  induce 
fatigue  in  skeletal  muscles.  Force  data  for  dynamic  force  experiments  was  captured  using  NI  ELVIS™  with 
Interlink™  Electronics  FSR  0.5”  circular  force  sensor. 


Fig.  2.2.2(n).  Experimental  Set-Up  for  Dynamic  Force  Levels. 
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Fig.  2.22(6).  Experimental  Set-Up  for  50  Pounds  Static  Force  Levels. 


3  Array  sensors 

3.1  Physical  development 

In  this  method  we  are  using  an  array  of  8  sensors  which  are  arranged  in  such  a  way  that  they  cover  the  maximum 
area  and  they  are  further  attached  to  the  sEMG  sensors  which  are  further  connected  to  the  Bagnoli  system.  The  best 
part  of  this  type  of  sensor  is  the  ease  of  application  but  the  drawback  is  the  crosstalk  which  hinders  the  collection  of 
EMG  signals.  But  the  results  which  we  got  during  tests  were  very  encouraging  and  removing  the  factor  of  crosstalk, 
it  will  be  of  great  importance  as  it  has  8  sensors  on  a  single  board  and  thus  the  area  under  observation  increases, 
without  increasing  the  number  of  sensors.  The  output  is  derived  the  same  way  as  it  is  taken  with  sEMG  sensors 
setup.  So  there  is  not  much  difference  in  the  technique  but  the  only  factor  is  the  number  of  sensors  that  a  board 
comprises. 


Fig.  3.1.1(a).  Array  Sensors  Connected  to  the  System. 


Fig.  3.1.1(b):  Array  Sensors  Tested  onNI  Lab  VIEW1 


Construction: 
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The  array  sensor  is  a  group  of  8  sensors  arranged  in  a  circular  manner  on  a  single  board,  the  surface  of  the  board 
should  be  something  that  prevents  cross-talks  between  signals.  So  after  choosing  the  surface  the  small  sensors  are 
placed  and  are  further  connected  to  the  Bengali  system  which  derives  signals  from  sEMG  signals  and  thus  this  is  the 
way  the  whole  set  up  is  made.  The  connection  between  sEMG  sensors  is  made  with  array  sensors  in  a  way  that  the 
output  from  them  is  easily  derivable  so  they  are  connected  with  multithread  wires  so  that  the  stable  connection  is 
maintained  and  the  signal  is  continuous. 

Scope: 

The  type  of  array  sensor  arrangement  shown  can  be  extremely  useful  as  the  area  under  the  observation  is 
increased  and  also  the  cost  of  this  is  very  low  as  compared  to  the  conventional  array  sensors.  So  if  the  problem  of 
crosstalk  is  removed  then  this  can  be  a  huge  step  as  it  will  not  only  help  us  to  get  excellent  results  but  also  at  the 
same  time  reduce  the  count  of  array  sensors  applied  as  it  has  8  sensors  on  board  as  compared  to  the  2  or  3  on  board 
sensors  available. 


3.2  Fusion  of  data 

In  this  part  of  the  research  work,  array  of  three  sEMG  sensors  as  shown  in  Fig.  2.2.2(a)  are  used  along  with  a 
proposed  sensor  fusion  scheme  that  results  in  a  simple  Multi-Input-Single-Output  (MISO)  transfer  function. 
Experimental  data  is  used  along  with  System  Identification  to  find  this  MISO  system.  A  Genetic  Algorithm  (GA) 
approach  is  employed  to  optimize  the  characteristics  of  the  MISO  system.  The  proposed  fusion-based  approach  has 
promising  results  discussed  in  simulation  results  section. 


Method  I: 

The  data  collected  from  multiple  sensors  are  processed,  estimated  using  sensor  fusion  technique.  A  data  fusion 
technique,  by  combining  the  data  from  different  sensors,  enables  one  to  achieve  more  specific  inferences  about  the 
measured  data.  The  sEMG  sensors  pick  up  the  cross-talk  from  muscle  fibers  adjacent  to  motor  units.  In  [P3],  sensor 
fusion  was  done  in  the  time  domain,  by  taking  sEMG  as  input  and  force  signal  as  the  output.  We  used  combinations 
of  different  filters  (Butterworth,  Chebyshev,  Exponential  and  Half-Gaussian  filters)  and  different  information 
criteria  based  on  Akaike  (AIC),  Bayesian  (BIC),  and  Kull-back  (KIC)  and  concluded  that  the  KIC  criterion  with 
Half  Gaussian  filtering  gives  the  best  EMG-Force  model  fit. 

In  the  present  work,  sensor  fusion  is  done  in  the  frequency  domain  for  the  sEMG  data.  Fig.  3.2(a)  gives  the  flow 
diagram  of  the  fusion  technique  used  based  on  the  GA  [P4]. 


u1 

^3 


Fig.  3.2(a).  Flow  Diagram  of  the  Fusion  Technique. 


The  data  from  the  three  sensors  around  the  individual  motor  unit  are  collected,  rectified  and  then  filtered  using  a 
Half-Gaussian  filter.  By  using  the  system  identification  technique  the  dynamic  relationship  between  the  sEMG 
datau1,u2,  and  from  the  three  sensors  and  their  corresponding  force  are  determined.  In  this  proposed  fusion 
algorithm,  SI  is  achieved  by  utilizing  an  Output  Error  (OE)  model  first  for  each  individual  data  set.  The  OE  model 
structure  is  given  as  follows: 
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yGt)  —  -  iifc)  4-  eft) 


(3.2.1) 

where  B,  and  fare  the  polynomials,  q  is  shift  operator,  s(0  is  output  error,  y(t)  is  system  output,  k  is  input,  nfc  is 
the  system  delay  and  t  is  time  index. 

The  corresponding  continuous-time  model  is  given  by  the  transfer  function  in  Equation  (3.2.2). 

(3.2.2) 


£i's'l  =  —  =  1;'P»h-lg:it&  ,;'+-  + Oi 


F(s)  1  +  ■■  ■  +/i 

Similar  to  the  discrete-time  case  the  orders  of  the  numerator  and  denominator  are  determined  by  nb  andnf.  For 
multi-input  systems,  nto  and  n/are  row  vectors.  are  the  coefficients  of  the  numerator  and  denominator 
polynomials  respectively. 

MISO  transfer  function  ( H )  is  constructed  by  using  the  poles  of  the  three  individual  identified  OE  models, 
corresponding  to  each  sensor,  while  the  denominators  of  the  respective  individual  transfer  functions  (corresponding 
to  each  OE  model)  are  transferred  over  to  the  new  MISO  transfer  function,  the  corresponding  zeros  are  found 
through  the  use  of  a  GA.  Generally  GA‘s  can  find  global  optimum  points  if  elitism  is  used  and  sufficient  number  of 
generations  is  allowed  in  the  algorithm.  This  optimization  algorithm  is  rather  computationally  expensive,  but  since 
there  was  not  a  computational  time  requirement,  one  is  free  to  use  GA  rather  than  other  intelligent  based  algorithms. 
Chromosomes  are  constructed  by  designating  each  zero  of  a  numerator  as  a  gene.  Since  a  discrete  time  model  is 
utilized,  the  search  area  is  limited  to  the  unit  circle  (and  the  resulting  MISO  model  is  decreased  to  be  minimum 
phase).  The  number  of  potential  zeros  was  set  to  the  order  of  the  corresponding  denominator  because  the  number  of 
zeros  is  at  most  the  number  of  poles  (for  a  causal  system,  can’t  predict  the  future).  The  objective  function  was  set  as 
the  error  squared  of  the  resulting  MISO  system  ( H )  can  be  constructed  as  follows: 

From  to  output  - 

sE  -  3.843s7  4  7,729s6  -10.78s5  +  10.6s4  -  7,417s3  4  3.603s2  -  0,9795 s  4  0,1192 

sE  -  4.028s7  4  6.325s6  -  4.121s5  -  1.545s4  4  5.87s3  -  5.433s2  4  2.28s  -  0.3496 
From  u2  to  output  - 

sE-  4.339s7  4  9.005s6  -  12.42s5  4  12.22s4  -  8.117s3  4  3.427s2  -  0.9134  s  4  0.1424 

sB-  4.028s7  4  6.325  s6-  4.121s5  -  1.545  s4  4  5.87s3  -  5.433  s2  4  2.28s  -  0.3496 
From  M  g  to  output  - 

s ~  -  3.522  s7  4  6.655  s6-  8.864s5  4  8,183  s4  -  5.365  s3  4  2.423s2-  0.557  s  4  0.09585 
sB-  4.028s7  4  6.325  s6-  4.121s5  -  1.545  s4  4  5.87s3  -  5.433  s2  4  2.28s  -  0.3496 

Method  II: 

In  this  method  also,  fusion  algorithm  was  done  in  frequency  domain.  First  sEMG  was  rectified  and  filtered  using 
Half-Gaussian  filter.  Three  output  error  models  were  obtained  using  the  system  identification  toolbox.  By  plotting 
the  bode  plots  of  three  models  separately  and  combining  the  peaks  of  all  three  models  using  the  fusion  algorithm 
proposed  in  method  I,  we  got  the  results  as  shown  in  the  following  section. 


Simulation  Results: 


Fig.  3.2(b).  Final  Fusion  Based  Force  Vs.  Measured  Force  Signal. 
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Fig.  3.2(c).  Estimated  Force  Output  for  Individual  OE  Models  and  the  Measured  Force  Signal. 
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Fig.  3.2(d).  Validation  Plot  for  Data  Set-I. 
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Fig.  3.2(e).  Validation  Plot  for  Data  Set-II. 
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Fig.  3.2(f).  Comparison  between  Measured,  Fusion  Based  and  Single  OE  Model  Output  Force. 
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Fig.  3.2(g).  Bode  Plots  of  Three  Different  Sensors. 
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Fig.  3.2(h).  Fusion  Based  Force  Signal. 


Conclusion: 

Both  the  methods  were  done  in  frequency  domain.  Method  I  gave  82  %  fit  value  and  Method  II  needs  to  be 
improvised.  In  future,  we  want  to  compare  the  three  different  fusion  techniques. 


4.  Skeletal  Muscle  Fatigue  and  Spectral  Analysis  of  sEMG  Signals 

As  it  is  explained  in  the  earlier  part  of  the  report  that  muscle  fatigue  is  a  complex  phenomenon  and  affects  the  sEMG 
signals  as  it  progress.  We  conducted  the  experiments  to  capture  the  muscle  fatigue  as  a  change  in  sEMG  as  the 
muscle  fatigue  progress.  sEMG  signals  are  analyzed  to  investigate  the  change  in  frequency  domain  during  skeletal 
muscle  fatigue  [P4].  Fig.  2.2. 2(n)  and  (o)  shows  the  experimental  set-ups  for  dynamic  and  static  force  experiments 
to  induce  skeletal  muscle  fatigue.  The  sEMG  signals  are  filtered  with  a  nonlinear  Teager-Kaiser  Energy  (TKE) 
operator-based  nonlinear  spatial  filter  [36].  Two  sets  of  dynamic  force  data  are  segmented  into  three  and  five  parts 
and  two  sets  of  static  force  data  are  segmented  into  five  parts  each.  There  is  a  75%  overlap  between  the  two  adjacent 
segments.  A  number  of  classical,  model-based  and  eigenvector  based  spectral  estimation  techniques  are  used  to 
study  the  change  in  the  sEMG  signals  as  a  result  of  muscle  fatigue.  In  classical  methods  Fast  Fourier  transform 
(FFT)  and  Welch‘s  averaged  modified  periodogram  methods  are  used.  In  case  of  model-based  methods  Yule-Walker 
(Y-W),  Burg,  Covariance  (Cov.)  and  Modified  Covariance  (Mcov.)  Autoregressive  (AR)  methods  are  applied.  For 
eigenvector  methods  Multiple  Signal  Classification  (MUSIC)  and  Eigenvector  (EIG)  spectral  estimation  methods 
were  selected  for  processing  sEMG  signals.  Using  these  spectrum  analysis  techniques,  Power  Spectral  Density 
(PSD)  estimates  and  detailed  documentations  of  the  sEMG  signals  were  obtained.  These  methods  were  compared  in 
terms  of  their  frequency  resolution  and  the  effects  in  determination  of  skeletal  muscle  fatigue. 

A.  Discrete  Fourier  Transform  (DFT): 

DFT  which  is  the  computational  basis  of  the  spectral  analysis  transforms  the  time  or  space  domain  data  into 
frequency  domain  data  [37].  The  DFT  of  a  vector  x  of  length  N  is  given  as 
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(4.A.1) 


=  If=i  *(/>*  Vj, 

where  mN  =  g(~7m^N  is  the  iV^  root  of  unity  [37]. 

B.  Welch ’s  averaged  modified periodogram  method: 

As  the  name  suggests,  the  =Wdch‘s  averaged  modified  periodogram  method4  depends  on  the  periodogram  of  the 
signal  which  is  given  by  Equation  (4.B.1),  [37]. 

■SjjarC/3  =  ^  Up  =  i  * 00  exp  (-j 2-xfn )  I2 .  (4.B.1) 

In  Welch  method,  the  signal  is  segmented  into  eight  parts  of  equal  length  with  an  overlapping  ratio  of  50%  and  each 
part  is  segmented  using  a  Hamming  window  as  given  by  Equation  (4.B.2),  [37]. 

u- (n)  =  0.54  —  0.46 cos  ,  0  <  n  <  N  .  (4.B.2) 

C.  Yule-Walker  (Y-W)  autoregressive  (AR)  method: 

The  Yule-Walker  autoregressive  method,  also  called  the  autocorrelation  method,  estimates  the  power  spectral 
density  (PSD)  of  the  input.  This  method  fits  an  autoregressive  (AR)  model  to  the  windowed  input  data  by 
minimizing  the  forward  prediction  error  in  the  least-squares  sense.  This  formulation  leads  to  the  Yule-Walker 
equations,  which  are  solved  by  Levinson-Durbin  recursion  [37]. 

D.  Burg  autoregressive  (AR)  method: 

The  Burg  autoregressive  (AR)  method  is  a  parametric  spectral  estimation  method  of  the  signal,  x.  The  power 
spectral  density  is  calculated  in  units  of  power  per  radians  per  sample.  This  method  is  based  on  the  minimization  of 
the  forward  and  backward  prediction  error  and  on  estimation  of  the  reflection  coefficients  [37]. 

E.  Covariance  (Cov.)  autoregressive  (AR)  method: 

The  covariance  autoregressive  (AR)  method  uses  the  covariance  algorithm  to  estimate  the  parametric  spectral 
density  of  the  signal,  x.  Based  on  causal  observation  of  the  input  signal,  the  covariance  method  minimizes  the 
forward  prediction  error  and  fits  an  AR  linear  prediction  filter  model  to  the  signal  [37]. 

F.  Modified  covariance  (Mcov.)  autoregressive  (AR)  method: 

Modified  covariance  autoregressive  (AR)  method  estimates  the  PSD  of  the  signal  using  the  modified  covariance 
method.  Based  on  the  causal  information  of  the  input  signal,  the  modified  covariance  method  fits  an  autoregressive 
(AR)  linear  prediction  filter  model  to  the  signal  by  simultaneously  minimizing  the  forward  and  backward  prediction 
errors.  The  spectral  estimate  returned  by  this  method  is  the  magnitude  squared  frequency  response  of  the  AR  model 
[37]. 

G.  Multiple  Signal  Classification  (MUSIC)  spectral  estimation  method: 

The  MUSIC  algorithm  estimates  the  pseudospectrum  (in  rad/sample)  at  the  corresponding  vector  of  frequencies  for 
the  input  signal  x  [37].  This  algorithm  uses  the  estimates  of  the  eigenvectors  of  a  correlation  matrix  associated  with 
the  input  signal  using  Schmidt's  eigenspace  analysis  method  [37].  The  MUSIC  pseudospectrum  estimate  is  given  by 
Equation  (4.G.1), 


,tf>  = 


tfXEELjH-i  Vfctf 


(4.G.1) 


where  Ar  is  the  dimension  of  the  eigenvectors  and  vk  is  the  k- th  eigenvector  of  the  correlation  matrix  [37].  The 
signal  subspace  has  a  dimension  p  and  the  eigenvectors  vk  used  in  the  sum  corresponds  to  the  smallest  eigenvalues 
and  also  spans  the  noise  subspace  [37].  The  vector  9(f)  consists  of  the  complex  exponentials,  so  the  inner  product 
vk  9(f)  amounts  to  a  Fourier  transform.  To  estimates  the  pseudospectrum,  the  squared  magnitudes  are  summed  for 
FFT  computed  for  each  vk  [37]. 

H.  Eigenvector  (EIG)  spectral  estimation  method: 

The  eigenvector  spectral  estimation  method  estimates  the  pseudospectrum  (in  rad/sample)  at  the  corresponding 
vector  of  frequencies  using  estimates  of  the  eigenvectors  of  a  correlation  matrix  associated  with  the  input  signal  x 
[37].  This  method  estimates  the  pseudospectrum  from  a  signal  or  a  correlation  matrix  using  a  weighted  version  of 
the  MUSIC  algorithm  derived  from  Schmidt's  eigenspace  analysis  method  [38,  39].  To  find  the  frequency  content  of 
the  signal  the  algorithm  performs  eigenspace  analysis  of  the  signal's  correlation  matrix.  Singular  value 
decomposition  is  used  to  compute  the  eigenvalues  and  eigenvectors  of  the  signal's  correlation  matrix  [37].  This 
method  computes  the  pseudospectrum  estimate  as  given  by  Equation  (4.H.1). 


pM  = 


(4.H.1) 


where  the  eigenvectors  have  a  dimension  of  Ar  and  vk  is  the  k- th  eigenvector  of  the  correlation  matrix  [37].  The 
signal  subspace  has  a  dimension  p  and  the  eigenvectors  vk  used  in  the  sum  corresponds  to  the  smallest  eigenvalues 
and  also  spans  the  noise  subspace  [37].  The  vector  9(f)  consists  of  the  complex  exponentials,  so  the  inner  product 
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vKe(f)  amounts  to  a  Fourier  transform  and  to  estimate  the  pseudospectrum,  the  squared  magnitudes  are  summed 
for  FFT  computed  for  each  vk  [37]. 

I.  Selection  of  Model  Orders  for  Model-Based  and  Eigenvector  Based  Methods: 

Model-based  and  eigenvector-based  methods  need  to  have  a  specific  model  order  which  is  an  important  aspect  of  the 
use  in  these  methods.  Using  the  sEMG  and  force  data  as  input  and  outputs  for  three  and  five  sets  of  the  segments  for 
different  data  sets,  we  created  model  structure  matrices  using  =struc‘  function  in  MATLAB®,  using  =arxstruc‘  we 
compared  a  model  order  of  1  to  50th  with  varying  delay  of  1  to  50  using  cross-validation  on  the  second  half  of  the 
data  set.  With  this  approach,  it  was  possible  to  select  the  order  that  gives  the  best  fit  for  the  validation  data  set. 

J.  Results  and  Discussion: 

sEMG  signal  changes  as  a  consequence  of  muscle  fatigue  [40-45],  the  amplitude  of  the  PSD  of  the  signal  increases 
and  the  median  frequency  shifts  towards  the  lower  frequency  range  [43-45].  In  this  study,  PSDs  of  the  different 
segments  of  each  sEMG  data  set  were  obtained  using  FFT,  Welches  averaged  modified  periodogram,  Yule -Walker, 
Burg,  Covariance,  Modified  Covariance  autoregressive  (AR),  Multiple  Signal  Classification  (MUSIC),  and 
Eigenvector  spectral  estimation  methods.  The  objective  of  this  study  was  to  determine  preferred  methods  of  signal 
processing  that  elevates  the  sensitivity  of  muscle  fatigue  as  represented  in  the  PSD  of  the  sEMG  signal.  An 
increased  sensitivity  allows  for  better  modeling  of  the  fatigue  phenomena  and  hence  more  accurate  sEMG  models. 
Ultimately  this  may  lead  to  better  prosthetic  control. 

Data  of  two  experiments  for  dynamic  force  variations  was  segmented  in  three  and  five  parts  respectively.  Each 
segment  is  with  an  overlap  of  75%  with  its  adjacent  segment.  Using  different  methods,  we  computed  the  PSDs  for 
each  segment.  For  the  dynamic  force  experiments,  the  maximum  value  of  PSDs  of  sEMG  signal  increases  with 
muscle  fatigue  as  time  or  segment  number  is  increased.  The  classical  methods  (FFT  and  Welch)  and  eigenvector 
based  methods  (MUSIC  and  Eigenvector  (EIG.))  are  representing  this  change  well  in  case  of  maximum  PSD  values 
and  show  a  clear  difference. 


TABLE  4.1 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  AND  EIGENVECTOR  BASED  METHODS  -  DYNAMIC 

VARYING  FORCE  -  EXPERIMENT  2 


Segment  No. 

Classical-Methods 

Eigenvector-Methods 

FFT 

Welch 

MUSIC 

EIG 

1st 

4.1e+6 

0.0082 

371.59 

1.0819 

2nd 

5.09e+6 

0.0100 

424.43 

1.3892 

3rd 

5.74e+6 

0.0111 

480.71 

1.4132 

4th 

6.79e+6 

0.0133 

508.64 

1.4936 

5th 

2.13e+7 

0.0379 

695.08 

10.2118 

Table  4.1  lists  the  peak  values  of  the  PSDs  of  five  segments  using  classical  and  eigenvector  based  methods  for  a 
dynamically  varying  force  experiment.  Fig.  4(a)  shows  the  overlapping  plot  of  PSDs  for  five  segments  using  the 
MUSIC  algorithm  based  spectral  estimation  method.  The  increase  in  the  maximum  PSD  value  is  evident  from  the  1st 
to  the  5th  segment  of  the  data. 


500  1000  1500  2000 

Frequency  (radian/sample) 


Fig.  4(a).  PSD  vs.  Frequency  Plot  for  MUSIC  Algorithm  -  Dynamic  Force  Experiment. 


Fig.  4(b)  shows  the  resulting  PSD  using  the  Burg  method.  Comparing  Fig.  4(a)  and  4(b),  the  progression  of  fatigue 
influence  shift  in  PSDs  is  evident  in  both  plots.  However,  the  MUSIC  algorithm  not  only  shows  larger  amplitudes, 
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but  also  a  greater  relative  sensitivity  to  fatigue.  The  rather  equal  spacing  between  the  lines  of  the  PSD  for  the 
MUSIC  algorithm  compared  to  the  Burg  method  indicates  a  rather  more  linear  relationship  of  the  fatigue 
progression. 
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Fig.  4(b).  PSD  vs.  Frequency  Plot  for  Burg  Method  -  Dynamic  Force  Experiment. 


TABLE  4.2 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  -  STATIC  FORCE  -  50  POUNDS  -  EXPERIMENT  1 


Segment  No. 

Classical-Methods 

Eigenvector-Methods 

FFT 

Welch 

MUSIC 

EIG 

1st 

1.03e+5 

6.81e-5 

1413 

9.93e-4 

2nd 

1.80e+5 

1.12e-4 

1793 

16e-4 

3rd 

3.21e+5 

1.80e-4 

2296 

28e-4 

4th 

4.25e+5 

2.62e-4 

3102 

37e-4 

5th 

6.23e+5 

3.58e-4 

7723 

104e-4 

TABLE  4.3 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  -  STATIC  FORCE  -  50  POUNDS  -  EXPERIMENT  1 


Segment  No. 

1st 

2nd 

3rd 

4th 

5th 

Model-Based 

Methods 

7.7e-7 

1.3e-6 

2.3e-6 

3.1e-6 

7.81-6 

The  sEMG  data  of  two  experiments  for  static  force  (50  pounds)  were  processed  and  the  maximum  PSDs  of  five 
segments  using  classical,  model-based,  and  eigenvector  based  methods  were  computed.  Data  from  both  the 
experiments  show  similar  results  as  the  dynamic  case.  Table  4.2  lists  the  peak  values  of  the  PSDs  of  five  segments 
using  classical  and  eigenvector  based  methods  for  static  force  (50  pounds)  for  one  experiment.  Table  4.3  lists  the 
maximum  values  of  the  PSD  for  model-based  methods:  Y-W,  Burg,  Covariance,  and  modified  covariance.  All  of 
these  methods  resulted  in  the  same  maximum  values  for  each  segment.  Comparing  Table  4.2  and  4.3,  we  recognize 
the  large  difference  in  maximum  value  between  the  model-based  methods  and  the  corresponding  values  from  the 
FFT  and  MUSIC  method.  Fig.  4(c)  shows  the  overlapping  plot  of  PSDs  for  five  parts  using  eigenvector  algorithm 
based  spectral  estimation  method.  The  increase  in  the  maximum  PSD  value  is  evident  from  the  1st  to  the  5th  segment 
of  the  static  force  sEMG  data. 

The  eigenvector  method  produces  a  similar  characteristic  as  the  MUSIC  algorithm  and  distinguishes  itself  by  also 
providing  a  more  linear  characteristic  of  the  fatigue  progression  and  a  greater  relative  sensitivity.  Fig.  4(d)  depicts 
the  PSD  generated  by  using  FFT  method  for  the  1st  and  5th  segments  of  a  static  force  experiment.  While  providing  a 
large  maximum  value,  the  FFT  method  is  limited  by  its  own  spectral  resolution  (1/Ar)  and,  due  to  windowing  of  the 
finite  data  set,  results  into  spectral  leaking. 


Page  42  of  531 


Eig  -  Static  Force  50  Pounds  -  Experiment  1 
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Fig.  4(c).  PSD  vs.  Frequency  Plot  for  Eigenvector  Method  -  Static  Force  of  50  Pounds. 


FFT  -  Static  Force  -  50  Pounds  -  1st  and  5th  Segment  -  Exp33 
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Fig.  4(d).  PSD  vs.  Frequency  Plot  for  FFT  Method  -  Static  Force  of  50  Pounds. 


All  the  model-based  methods  for  both  dynamic  and  static  force  levels  produce  the  same  peak  values  and  the  same 
PSD  for  the  corresponding  experiment.  Since  Burg  and  Y-W  methods  guarantee  stability  while  the  covariance  and 
modified  covariance  methods  have  conditions  for  stability  to  be  satisfied  (i.e.  min.  order  must  be  of  certain  length  of 
the  input  frame  size),  Burg  and  Y-W  should  be  the  preferred  methods  for  sEMG  analysis.  However,  the  Burg 
method  is  to  be  preferred  if  short  data  sets  are  used. 

Comparing  the  eigenvector  based  methods  (MUSIC  and  Eigenvector),  both  of  these  methods  are  frequency 
estimator  techniques  based  on  eigenanalysis  of  the  autocorrelation  matrix  where  the  resulting  estimate  has  sharp 
peaks  at  the  frequencies  of  interest.  The  eigenvector  method  uses  inverse  eigenvector  weighting  whereas  the  MUSIC 
method  uses  unity  weighting,  implying  that  the  eigenvector  method  gives  fewer  spurious  peaks  than  the  MUSIC 
algorithm  [46].  As  seen  from  the  dynamic  experiment  results,  the  MUSIC  method  provides  for  a  better  spacing 
between  the  segments  based  PSDs  compared  to  the  Burg  and  eigenvector  method.  From  static  experiments,  we 
conclude  that  all  three  (MUSIC,  Burg,  and  Eigenvector)  methods  perform  similarly  if  a  linear  relationship  of  the 
fatigue  progression  is  desired. 

Conclusion  and  Future  Work 

In  this  part  of  the  research,  we  characterized  muscle  fatigue  using  a  PSD  representation  of  different  segments  of 
sEMG  data.  Classical  (fast  Fourier  transform  and  WelclTs  averaged  modified  periodogram),  model-based  (Yule- 
Walker,  Burg,  Covariance,  and  Modified  Covariance  autoregressive  (AR)  method)  and  eigenvector  based  methods 
(Multiple  Signal  Classification  (MUSIC)  and  eigenvector  spectral  estimation  method)  are  used  to  compute  the 
PSDs.  Classical  and  eigenvector  based  methods  are  more  sensitive  than  the  model-based  methods  for  analyzing  the 
fatigue  related  changes  in  sEMG  signal.  However,  the  MUSIC  algorithm  provides  good  maximum  value  in  the  PSD 
as  well  as  a  clear  distinction  between  the  segmented  sEMG  data.  The  latter  point  is  indicative  of  a  relative  linear 
fatigue  progression  in  time  for  the  same  case  when  the  MUSIC  algorithm  is  utilized.  In  the  future  work  these  results 
can  be  used  to  design  and  improve  the  skeletal  muscle  Jorce-sEMG-Fatigue4  based  models  [P4]  for  prosthetic 
design  and  other  rehabilitation  research. 

5.  Force  models 

To  model  the  skeletal  muscle  force  and  sEMG  signals  we  tried  all  the  possible  models  available  in  MAT  LAB  ^ .  For 
the  data  set  acquired  using  the  experimental  set-up  in  Fig.  2(a). 
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5.1  Regular  force  model 

5.1.1  Skeletal  Muscle  Force  Estimation  Using  Nonlinear  Models  for  an  array  of  three  Semg  sensors 

We  tried  multi  nonlinear  Auto  Regressive  exogenous  (ARX)  and  Wiener-Hammer  stein  models  with  different 
nonlinearity  estimators/classes  using  System  Identification  (SI)  for  three  sets  of  sensor  data.  For  the  motor  point, 
ring  1  and  ring  2  sensors  we  obtained  seven,  seven  and  eight  nonlinear  models  respectively  [P5,  P6].  First,  the 
outputs  of  these  nonlinear  models  are  fused  separately  and  then  the  final  fusion  based  outputs  from  three  sensors  are 
fused  further  to  have  better  force  estimates  for  an  array  of  three  sEMG  sensors.  Data  fusion  is  done  using  an 
adaptive  KIC  probability. 

Results  and  Discussion 

This  section  deals  with  the  results,  discussion  and  future  work.  The  following  plots  show  the  nonlinear  (ARX  and 
Wiener-Hammerstein)  model  and  adaptive  fusion  algorithm  based  estimated  force  output  for  each  sensor  first  and 
then  finally  combined  adaptive  fusion  based  output  for  all  three  sensors.  Fig.  5.1.1(a)  shows  the  overlapping  plot  of 
the  original  and  adaptive  fusion  based  force  output  for  the  motor  point  sensor.  The  output  is  the  result  of  the 
adaptive  fusion  algorithm  on  three  nonlinear  ARX  and  four  nonlinear  Wiener-Hammerstein  models  for  the  motor 
point  sensor  signal.  Fig.  5.1.1(b)  shows  the  overlapping  plot  of  the  original  and  adaptive  fusion  based  force  output 
for  ringl  sensor.  This  output  is  the  result  of  adaptive  fusion  algorithm  of  three  nonlinear  ARX  and  four  nonlinear 
Wiener-Hammerstein  models  for  ringl  sensor  signal.  Fig.  5.1.1(c)  shows  the  overlapping  plot  of  the  original  and 
adaptive  fusion  based  force  output  for  ring2  sensor.  This  output  is  the  result  of  adaptive  fusion  algorithm  on  three 
nonlinear  ARX  and  five  nonlinear  Wiener-Hammerstein  models  for  ring2  sensor  signal. 


Original  Force  and  NL-ARX  -  NL-HW-2-3-4-5  Models  Fused  Output  -  Sensor  Motor  Point 
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Fig.  5.1.1(a):  Original  and  Fusion  Based  Output  for  Motor  Point  Sensor. 


Original  Force  and  NL-ARX  -  NL-HW  -  Models  Fused  Force  Output  -  Ringl  Sensor 


Fig.  5.1.1(b):  Original  and  Fusion  Based  Output  for  Ringl  Sensor. 
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Fig.  5.1.1(c):  Original  and  Fusion  Based  Output  for  Ring2  Sensor. 

Fig.  5.1.1(d)  shows  the  overlapping  plot  of  the  original  and  final  combined  adaptive  fusion  based  force  output  for 
motor  point,  ringl  and  ring2  sensors.  The  output  is  the  result  of  adaptive  fusion  algorithm  on  the  final  outputs  of 
three  sensors  i.e.  motor  point,  ringl  and  ring2  as  shown  in  Fig.  5.1.1(a)  to  5.1.1(c).  Fig.  5.1.1(d)  shows  the  best 
skeletal  muscle  force  estimate,  which  is  the  result  of  the  multi  nonlinear  ARX  and  Wiener -Flammer stein  models  and 
an  adaptive  hybrid  data  fusion  on  these  nonlinear  models. 


Original  Force  and  NL-ARX  -  N I  -HW  Models  Output  -  MotorPoint  -  Ringl  -  Ring2  Sensors 
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Fig.  5.1.1(d):  Final  Plot  -  Original  and  Fusion  Based  Output  for  All  Three  Sensors. 


Fig.  5.1.1(e)  shows  the  error  plot  of  the  original  and  best-estimated  model  output  for  the  motor  point  sensor.  Fig. 
5.1.1(f)  shows  the  error  plot  of  original  and  final  multi  nonlinear  modeled  and  adaptive  hybrid  data  fusion  based 
force  estimate  (results  from  three  sensors,  nonlinear  modeling  and  adaptive  data  fusion  algorithm).  If  we  compare 
Fig.  5.1.1(e)  and  5.1.1(f),  it  is  very  clear  and  conspicuous  that  the  error  has  decreased  remarkably  and  is  very  close 
to  zero. 


Fig.  5.1.1(e):  Error  Plot  -  Original  and  Best-Estimated  Model  Output  for  Motor  Point  Sensor. 
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Final  Error  Plot  -  Original  and  Fused.  Output  -  MotorPoint,  Ring  1  andRing2  Sensors 


Fig.  5.1.1(f):  Final  Error  Plot  -  Original  and  Fusion  Based  Output  for  Motor  Point,  Ringl  and  Ring2  Sensors. 


5.1.2  Skeletal  Muscle  Force  Estimation  Using  Linear  and  Nonlinear  Models  for  an  array  of  three  sEMG 
sensors 

Once  we  tried  all  the  possible  nonlinear  models  available  in  MAT  LAB to  estimate  the  skeletal  muscle  force  for 
three  sensor  array,  we  thought  of  using  linear  and  nonlinear  models  together  to  model  the  relationship  between  the 
skeletal  muscle  force  and  sEMG  [P7,  P8].  First  we  tried  this  approach  for  the  motor  point  sensor  and  then  we  tried 
this  for  the  all  three  sensors  as  shown  in  the  experimental  set-up  in  Fig.  2.2.2(a).  For  three  sensor  array  data  set  we 
obtained  multi  linear  and  nonlinear  models  for  motor  point,  ring  1  and  ring  2  data  sets.  For  motor  point,  ring  1  and 
ring  2  sensors  we  obtained  =six  linear  and  three  nonlinear  models/  Jhree  linear  and  five  nonlinear  models/  =six 
linear  and  five  nonlinear  models/  respectively. 

Results  and  Discussion 

An  adaptive  KIC  probability  based  data  fusion  algorithm  is  applied  to  linear  and  nonlinear  models  separately  for  the 
models  obtained  using  three  sets  of  input  and  output  data  for  three  sEMG  sensors.  First  the  fusion  based  output  for 
each  sensor  is  obtained  and  then  the  three  fusion  based  outputs  from  three  sEMG  sensors  are  fused  to  get  the 
improved  estimates  of  the  skeletal  muscle  force.  The  following  plots  show  the  improvement  in  the  force  estimates  in 
succession  with  different  sensors  separately  and  then  combined.  All  the  plots  have  the  measured  skeletal  muscle 
force  signal  in  green  and  the  estimated  force  signal  in  red  color.  Fig.  5.1.2(a)  is  a  plot  of  measured  and  Chebyshev 
type  II  filtered  finger  force  data  with  the  adaptive  KIC  probability  fusion  based  force  for  the  first  sensor  named  as 
ringl.  The  two  signals  are  very  clear  and  the  estimated  signal  shows  good  follow  up  of  the  measured  signal.  Fig. 
5.1.2(b)  shows  the  measured  and  Chebyshev  type  II  filtered  skeletal  muscle  force  with  the  adaptive  KIC  probability 
fusion  based  force  signals  for  the  second  sensor  named  as  ring2.  It  is  evident  that  the  measured  signal  has  a  very 
close  follow  up  by  the  estimated  signal.  Fig.  5.1.2(c)  shows  the  measured  and  Chebyshev  type  II  filtered  skeletal 
muscle  force  with  the  adaptive  KIC  probability  fusion  based  force  signals  for  third  sensor  placed  on  the  motor  point 
of  ring  finger.  The  results  for  this  sensor  are  best  among  the  three  based  of  the  used  three  sensor  data  and  it  is 
evident  that  the  measured  signal  has  a  very  close  follow  up  by  the  estimated  signal. 


Chebyshev  Type  II  Filtered  Vs.  Linear-NL  Models  -  Adaptive  KIC  Probability  -  Ringl 


Fig.  5.1.2(a).  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability  Based  Force  from  Linear  and  Nonlinear  Models  for 

Ringl  Sensor. 
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Chebyshev  Type  II  Filtered  Vs.  Linear-NL  Models  -  Adaptive  KIC  Probability  -  Ring2 


Fig.  5.1.2(b).  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability  Based  Force  from  Linear  and  Nonlinear  Models  for 

Ring2  Sensor. 


Chebyshev  Type  II  Filtered  Vs.  Motor  Point  -  Linear-NL  Models  -  Adaptive  KIC  Probability 


Time  (60  Seconds)  x104 

Fig.  5.1.2(c).  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability  Based  Force  from  Linear  and  Nonlinear  Models  for 

Motor  Point  Sensor. 

Finally  the  estimated  skeletal  muscle  force  from  three  sEMG  sensors  is  further  fused  with  adaptive  KIC  probability 
based  data  fusion  algorithm.  Fig.  5.1.2(d)  shows  the  comparison  of  the  measured  and  Chebyshev  type  II  filter  with 
the  final  adaptive  KIC  probability  based  fusion  skeletal  muscle  force  estimate  using  three  sEMG  sensors.  The  final 
result  for  three  sensors  is  the  best  estimate  of  skeletal  muscle  force  and  it  is  evident  that  the  measured  signal  has  a 
very  close  follow  up  by  the  estimated  signal. 


Chebyshev  Type  II  Filtered  Ys.  Ringl-Ring2-MotorPoint  -  Linear-NL  Models  -  Adaptive  KIC  Probability 


Fig.  5.1.2(d).  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability  Based  Force  from  Linear  and  Nonlinear  Models  for 

Ringl,  Ring2  and  Motor  Point  Sensor  Combined. 

The  results  show  that  there  is  a  decrease  in  the  percentage  error  from  the  linear  and  nonlinear  model  fusion  based 
separate  outputs  to  the  fusion  based  combined  output  of  linear  and  nonlinear  models  combined  for  all  three  sensors. 
However,  these  results  show  that  there  is  a  16%  improvement  in  the  mean  fit  value  of  the  motor  point  signal  models 
with  the  adaptive  KIC  probability  based  data  fusion  algorithm  for  multi-sensors  (three  in  this  case).  Fig.  5.1.2(e)  and 
(f)  show  the  validation  plots  of  this  approach  for  two  separate  sets  of  sEMG  and  skeletal  muscle  force  data  where 


Page  47  of  531 


green  signal  is  the  Chebyshev  type  II  filtered  measured  force  signal  and  the  blue  signal  is  estimated  using  this 
approach. 


Validation  Plot  -  Separate  Experiment  Data  1 


Time  (60  Seconds)  x104 

Fig.  5.1.2(e).  Validation  Plot  1  -  Chebyshev  Type  II  vs.  Estimated  Force  Signal  using  Adaptive  KIC  Probability. 


Validation  Plot  -  Separate  Experiment  Data  2 


Fig.  5.1.2(f).  Validation  Plot  2  -  Chebyshev  Type  II  vs.  Estimated  Force  Signal  using  Adaptive  KIC  Probability. 

5.1.3  Adaptive  Neuro  Fuzzy  Inference  Systems  (ANFIS)  and  Smoothing  Spline  Curve  Fitting  Based 
Estimation  of  Skeletal  Muscle  Force 

Another  method  that  we  tried  to  estimate  the  skeletal  muscle  force  using  the  sEMG  signal  is  =ANFIS  and  Smoothing 
Spline  Curve  Fitting.4  Here  we  used  the  data  set  captured  using  an  array  of  nine  sEMG  sensors  as  shown  in  the 
experimental  set-up  of  Fig.  2.2.2(f).  Eight  Takagi-Sugeno-Kang  ANFIS  with  different  membership  functions  each 
with  a  rule  base  of  ten  are  designed.  The  training  of  each  ANFIS  is  done  using  a  hybrid  optimization  method  with 
zero  error  tolerance  and  40  epochs.  Each  of  the  force  signals  simulated  using  these  ANFIS  and  measured  force 
signals  are  applied  to  a  smoothing  spline  curve  fitting  method  [P9].  The  outputs  from  the  smoothing  spline  curve 
fitting  are  supplied  to  a  data  fusion  algorithm  with  adaptive  KIC  probability.  The  final  output  of  this  approach  gives 
improved  estimates  of  the  skeletal  muscle  force. 

Results  and  Discussion 

This  method  of  skeletal  muscle  force  estimation  gives  good  estimation  of  the  skeletal  muscle  force.  A  data  set  of  45 
seconds  is  used  to  estimate  the  output.  Fig. 5. 1.3(a)  shows  the  measured  and  estimated  skeletal  muscle  force  using 
this  method. 


Page  48  of  531 


ANFIS  -  CURVE  FITTING  -  DATA  FUSION  OUTPUT 


Fig.  5.1.3(a).  Plot  of  Measured  and  Proposed  Method  Estimated  Skeletal  Muscle  Force. 


Fig.  5.1.3(b)  and  (c)  are  the  validation  plots  for  45  seconds  and  60  seconds  of  data  sets  respectively.  All  of  these 
three  plots  show  a  very  close  follow  up  of  the  measured  signal  by  the  estimated  signal  using  this  approach.  Fig. 
5.1.3(d)  shows  the  percentage  error  between  the  measured  force  signal  and  estimated  finger  force  signal  for  45 
seconds  of  data  set.  The  percentage  error  reduced  to  the  lower  values  and  the  maximum  percentage  error  value  is  1.7 
percent.  The  mean  fit  value  between  the  measured  and  estimated  output  is  91.3  percent. 


Validation  Plot  1  -  ANFIS  -  CURVE  FITTING -DATA  FUSION 


Fig.  5.1.3(b).  Validation  Plot  1  Using  Different  Data  Set. 


Fig.  5.1.3(e)  shows  the  percentage  error  plot  for  different  data  set  of  60  seconds  duration.  Percentage  error  reduced 
to  lower  values  and  the  maximum  percentage  error  value  is  3.7  percent.  The  mean  fit  value  between  the  measured 
and  estimated  output  is  40  percent.  The  mean  fit  value  for  another  45  seconds  of  data  set  is  73  percent. 

Validation  Plot  2  -  ANFIS -CURVE  FITTING -DATA  FUSION 
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Fig.  5.1.3(c).  Validation  Plot  2  Using  Different  Data  Set. 
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Percentage  Error 


Fig.  5.1.3(d).  Percentage  Error  Plot. 


Percentage  Error  -  Validation  2  Data 


Time 

Fig.  5.1.3(e).  Percentage  Error  Plot  for  60  Seconds  Data  Set. 


5.2  Force-fatigue  model 

As  it  is  explained  in  earlier  part  of  the  report  that  muscle  fatigue  is  a  natural  and  dynamic  phenomenon  and  cannot 
be  avoided.  So,  it  is  equally  important  that  while  estimating  the  skeletal  muscle  force  from  sEMG  we  address  the 
issue  of  muscle  fatigue  or  have  some  compensation  for  the  change  in  sEMG  as  a  result  of  muscle  fatigue.  This  part 
of  the  research  deals  with  the  issue  of  skeletal  muscle  fatigue  and  try  to  design  the  mathematical  model  that  can 
capture  the  dynamics  of  muscle  fatigue.  Here  we  used  the  experimental  set-up  shown  in  Fig.  2.2.2(a),  and  the  data  is 
captured  for  a  muscle  fatiguing  contraction  for  the  dominant  hand  of  a  healthy  subject.  The  data  is  pre-processed 
using  nonlinear  Half-Gaussian  Bayesian  filter.  The  data  was  segmented  into  five  parts  with  91  percent  overlap 
among  the  adjacent  data  segments  for  three  sEMG  sensors.  Output  Error  (OE)  models  for  each  segment  is  obtained 
and  the  state  space  matrices  are  transformed  using  Modal  transformation.  The  eigenvalues  are  the  transformed 
matrices  are  plotted  and  curve  fitted  to  capture  the  dynamics  of  muscle  fatigue.  Finally  using  this  method  we 
obtained  three  discrete-time  dynamic  state  space  models  for  three  sEMG  sensors  for  the  ring  finger  [P10]. 

Results  and  Discussion 

Based  on  previous  research  results  and  after  trying  different  model  orders  for  sEMG/Force  data,  an  optimal 
model  order  of  =  2,-^  =1  is  used  in  this  work.  Equation  (5.2.1),  (5.2.2),  and  (5.2.3)  give  three  resulting 

discrete  time  state-space  sEMG-Force-Fatigue  models  for  motor  point  sensor,  sensor- 1  and  sensor-2  respectively. 

XfrXh  +  1)  =  AmxmGD  + 


¥m{h)  =  CmxmQd  +  D^EMCM 

(5.2.1) 

xL(h  +  1)  =  A±x±00  +  BLEMGLQi); 

i\(h)  =  CjijM  4-  D1EMGi_ (A). 

(5.2.2) 

x2 ih  +  1}  =  A2  x2 GO  +  B2EMG2  GO ; 

?2 GO  =  C2x20i)  -\-D2EMG2(h\ 

(5.2.3) 
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In  these  case  matrices  Dm,Dv  and  Dz  are  zero  matrices,  whereas  matrices  Am,Bmf  CmfAvBv  ClfA2,B2,  and  C2  are 


given  as: 


0 

Oml 


]■■  b-=[ 


LilmiCO 


’LL 

0 


*L2 


Bl  =  [^1L]i  B l  =  [c1L  A2  =  ^  ]  ;  B2  =  [^21^]  !  and  C2  =  [c31Ct)  c^Ct}];  where  the  elements  of 

matrices  Am,  Bm,  Cm,A2,  B2,  and  C2  are  function  of  time  and  the  elements  of  matrices  AVBL,  and  CL  are  constants 
for  this  particular  data  set.  The  time  variables  cvl1  (t) ,  CO,  bml  CO ,  b^  CO ,  cmLC0 ,  CO , 
q2LC0,  &22  CO,  2 l CO ^ CO ^  c31CO.  and  c22(t]  are  given  by  quadratic  polynomials  obtained  by  curve  fitting  the 
tracked  modal  transformed  coefficients.  These  time  variables  are  given  as: 

amL CO  =  Q.QQQ26r2  -  0.033 t  -  11; 
aTTO.  CO  =  -0.0018  t2  +  0.24t  -  33; 

bml(t)  =  — 0.002 12  +  0.33t  -h42;hm2(t)  =  —0.0017 12  +-  0.33t  +-  25; 
cml(0  =  C3.2  *  10"7]t2  -  C4.2  *  10“5)t  +  0.71; 
c^CO  =  (4.6  *  10”®)t2  -  (5.0  *  10_6]t  -  0.71; 
a2l(t)  =  -(1.2  -  10“5>£2  -h  0.0013 1  -  0.47; 

022  (t)  =  0.001 12  -  G.037£  -  35; 
b2l( 0  =  (6.4  *  10“*)t2  -  O.OOlt  +  0.3; 

£p 2 2 CO  =  0.0023 12  -  0.1 2t  -  75; 

c2L(t)  =  -Cl. 3  *  10"7)t2  -|-  (6.4  *  10_6)t  4-  0.71;  and  c22(t)  =  -(2.1*  10 ~9)t2  4(2.4*  IQ"7)  t  -  0.71. 
Elements  of  matrices  Av  Bl9  and  CL  are  constants  for  this  particular  data  set,  they  are  given  as: 


Q-n  =  —0.0159;  o12  =  —43.1033;  bn  =  0.0090;  bt2  =  —72.1075;  cn  =  0.7147;  and  cu  =  —0.7071. 


The  discrete  models  have  a  sampling  time  of  At  =  0.0005  seconds. 

Fig.  5.2(a)  and  (b)  shows  the  measured  and  estimated  fused  output,  depending  on  the  error  between  the  two  later 
follows  the  measured  signal  very  closely.  This  approach  identifies  a  fused  fatigue  model  and  estimates  an  adaptive 
probability  based  output  that  follows  the  measured  output  very  closely.  The  outcome  of  this  research  can  be  used  to 
estimate  the  skeletal  muscle  force  of  a  human  hand  for  prosthetic  hand  design,  application  and  improvement. 


Measured  Output  Ys.  Estimated  Fused  Output  with  Adaptive  Probability 


Fig.  5.2(a).  Measured  Output  Vs.  Estimated  Fused  Output  with  Adaptive  Probability. 
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Validation  Plot  -  Measured  Output  Vs.  Estimated  Fused  Output  With  Adaptive  Probability 


Time  (37.36  secs) 


Fig.  5.2(b).  Validation  Plot  -  Measured  Vs.  Estimated  Fused  Output  with  Adaptive  Probability. 


6.  Motion  model 

This  part  of  the  report  presents  a  novel  approach  to  dynamic  modeling  and  estimation  of  the  angles  of  the 
proximal  interphalangeal  (PIP)  joint  of  the  index  finger  with  the  corresponding  sEMG  signal.  This  can  be  used  as  an 
angle  control  signal  for  the  controller  and  replace  the  need  of  pre-programmed  motion  sets,  which  are  initiated  when 
some  threshold  value  of  the  measured  sEMG  signal  is  reached.  Here  we  are  using  System  Identification  (SI)  in  order 
to  obtain  a  dynamic  finger  angle  model/estimation.  The  data  for  the  simulation  and  modeling  is  captured  using  an 
experimental  set-up  as  shown  in  Fig.  2.2.2(n)  that  we  designed  for  this  specific  purpose. 


Index  Finger  Raw  sEMG  Signal 


Fig.  6(a).  Raw  and  Spatially  Filtered  sEMG  Signal  from  Index  Finger. 


Measured  Angle  Theta  -  Index  Finger 


Fig.  6(b).  Raw  and  Chebyshev  Type  II  Filtered  PIP  Joint  Angle  for  Index  Finger. 

The  recorded  sEMG  signal  for  the  index  finger  is  filtered  using  nonlinear  spatial  filtering  for  nine  channel 
surface  EMG.  The  nonlinear  spatial  filter  is  obtained  from  the  -Nonlinear  Teager-Kaiser  Energy  (TKE)  Operator,” 
[25].  Fig.  6(a)  shows  a  comparison  between  the  measured  and  nonlinear  spatial  filtered  sEMG  signals.  A  Chebyshev 
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type  II  infinite  impulse  response  (HR)  low  pass  filter  with  a  550  Hz  pass  band  frequency  is  used  to  filter  the  PIP 
joint  angle  data.  Fig.  6(b)  shows  the  comparison  between  the  measured  and  Chebyshev  type  II  filtered  skeletal 
muscle  force  signal. 

The  filtered  signals  are  smoothed  using  a  smoothing  spline  curve  fitting.  The  smoothed  sEMG  data  is  used  as 
input  and  the  respective  smoothed  finger  angle  data  is  used  as  output  for  a  system  identification  routine  to  obtain 
multiple  linear  and  nonlinear  models.  Five  linear  and  three  nonlinear  models  are  obtained  and  to  achieve  better 
estimates  of  the  finger  angles,  an  adaptive  probabilistic  Kullback  Information  Criterion  (KIC)  for  model  selection 
based  data  fusion  algorithm  is  applied  to  the  linear  and  nonlinear  model4 s  outputs.  Final  fusion  based  output  of  this 
approach  results  in  improved  estimates  of  finger  angles  [PI  1]. 

Results  and  Discussion 

The  sEMG  signals  from  the  sensor  array  and  angle  signals  of  PIP  joint  of  index  finger  are  filtered;  smoothing 
spline  curve  fitted  and  modeled  using  multiple  linear  and  nonlinear  models.  First  the  outputs  of  linear  and  nonlinear 
models  are  fused  separately  and  then  the  resultant  outputs  are  fused  using  an  adaptive  KIC  based  probability.  This 
approach  gives  improved  estimates  of  the  finger  angles  of  the  PIP  joint  of  the  index  finger  of  the  dominant  hand  of  a 
healthy  subject.  Results  are  presented  in  the  following  figures. 


Time 


Fig.  6(c).  Curve  Fitted  Vs.  Data  Fusion  Based  Angle  Using  Linear  Models. 


Curve  Fitted  Angle  Vs.  Data  Fusion  Using  Nonlinear  Models 


Fig.  6(d).  Curve  Fitted  Vs.  Data  Fusion  Based  Angle  Using  Nonlinear  Models. 

Fig.  6(c)  shows  the  overlapping  plot  of  the  measured-curve  fitted  angle  vs.  data  fusion  based  angle  using  linear 
models.  Fig.  6(d)  shows  the  overlapping  plot  of  the  measured-curve  fitted  angle  vs.  data  fusion  based  angle  using 
nonlinear  models.  Fig.  6(e)  shows  the  overlapping  plot  of  the  curve  fitted  angle  vs.  data  fusion  based  angle  using 
linear-nonlinear  models.  It  is  clearly  evident  that  the  data  fusion  based  output  follows  the  measured -curve  fitted 
output.  Mean  percentage  error  of  the  linear,  nonlinear  and  linear-nonlinear  models  data  fusion  based  angles  are 
2.5191e-005,  -4.5807e-006,  and  8.1167e-006  respectively. 


Page  53  of  531 


2.2 


Curve  Fitted  Angle  Vs.  Data  Fusion  Using  Linear  -  NL  Models 


Fig.  6(e).  Curve  Fitted  Vs.  Data  Fusion  Based  Angle  Using  Linear-Nonlinear  Models. 

Fig.  6(f)  shows  the  validation  plot  for  a  different  experimental  data  where  the  measured-curve  fitted  and  the 
linear-nonlinear  modeled  data  fusion  based  angle  signal  shows  a  close  follow  up.  Mean  percentage  error  of  the 
linear,  nonlinear  and  linear-nonlinear  models  data  fusion  based  angles  are  2.8202e-005,  3.3191e-005,  and  6.8079e- 
006  respectively. 


Validation  Plot  1  -  Curve  Fitted  Angle  Vs.  Data  Fusion  Based  Angle 


Fig.  6(f).  Validation  Plot  -  Curve  Fitted  Vs.  Data  Fusion  Based  Angle  Using  Linear-Nonlinear  Models. 

In  addition  to  the  research  conducted  on  the  sEMG-force  and  sEMG-angle  modeling,  we  also  investigated 
various  optimization  methods  for  improving  the  non-linear  Hammerstein  model  fits  for  sEMG-force  models.  This 
research  was  published  in  2010  [PI 5].  -A  Study  on  Hybridization  of  Particle  Swarm  and  Tabu  Search  Algorithms 
for  unconstrained  Optimization  and  Estimation  Problems.” 

Also,  we  learnt  a  great  deal  about  the  Hammerstein- Wiener  models  from  our  previous  research  which  led  to  the 
classification  of  amputation  utilizing  various  spatial  filters.  This  research  was  published  in  2010  [PI 6]. 
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IGoal  2A:  Intelligent  Controll 

The  overall  research  is  accomplished  under  four  goals:  Goal  1:  EMG  Signal  Identification  and 
Estimation,  Hand  Motion  Estimation,  Goal2:  Intelligent  Control  of  Prosthetics,  Goal  3:  Robotic 
Hand  and  Goal  4:  Biocompatibility  and  Signaling. 

The  schematic  diagram  of  the  proposed  work  is  shown  in  Figure  1  (see  the  works  of  Light  et  al. 
(2002);  Rodriguez-Cheu  and  Casals  (2006);  Zhao  et  al.  (2006)).  The  overall  system,  in  brief,  consists 
of  electromyographic  (EMG)  signal  acquisition  from  user  arm  for  surface  or  implanted  electrodes 
(in  the  implanted  case  we  focus  on  biocomptibility  based  on  nano-materials  research).  The  EMG 
signal  is  then  processed  for  feature  extraction  and  classification  or  identification  of  EMG  signal  to 
correspond  to  different  motions  of  the  prosthetic  hand.  The  classified  signal  is  then  used  to  control 
the  prosthetic  hand  using  actuators  and  driving  mechanisms. 


Biomaterials  & 
Biocompatibility 


1 

Us 

f 

;er 
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Arm 

EMG  Signal 
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Sensory  Feedback 

Visual  Feedback 


Figure  1:  Schematic  Diagram  of  Prosthetic  Hand  Technology 

Soft  computing  (SC)  or  computational  intelligence  (Cl)  (Konar  (2005))  is  an  emerging  field 
based  on  synergy  and  seamless  integration  of  neural  networks  (NN),  fuzzy  logic  (FL)  and  genetic 
algorithms  (GA)  (Karray  and  De  Silva  (2004)).  The  previous  works  on  prosthetic  hand  used  arti¬ 
ficial  neural  networks  by  Hudgins  (1991);  Christodooulu  and  Pattichis  (1999);  Light  et  al.  (2002), 
fuzzy  logic  by  Weir  and  Ajiboye  (2003);  Chan  et  al.  (2000);  Ajiboye  and  Weir  (2005),  genetic  algo¬ 
rithms  in  (Fernandez  et  al.  (2000))  etc  mostly  for  EMG  signal  classification  for  various  movements 
or  functions  of  the  prosthetic  hand. 


Goal  2:  Intelligent  Control  of  Prosthetics 

The  research  accomplishments  under  Goal  2A:  Intelligent  Control  of  Prosthetics  are 

1.  a  comprehensive  literature  updated  on  the  topic  of  control  techniques  for  prosthetic  hands 
was  conducted, 

2.  trajectory  planning,  kinematics  and  dynamics  of  a  prosthetic  hand  with  a  two-link  thumb 
and  four  three-link  fingers  based  on  analytical  models  were  investigated, 

3.  hard  control  techniques  (feedback  linearization,  PD,  PID,  optimal  and  adaptive)  to  prosthetic 
hand, 
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4.  soft  computing  strategies  (neural  networks,  fuzzy  logic,  adaptive  neural-fuzzy  inference  system 
and  genetic  algorithm)  to  prosthetic  hand,  and 

5.  fusion  of  hard  and  soft  control  techniques  (fuzzy  logic+PD  and  genetic  algorithm+PID)  to 
the  five-fingered  smart  prosthetic  hand. 

Overview  of  Control  Strategies  for  Smart  Prosthetic  Hand  Tech¬ 
nology 

In  research  and  development  of  a  prosthetic  hand  using  non-invasive  techniques,  the  three  basic 
problems  are,  first,  electromyographic  (EMG)  signal  acquisition,  and  classification  to  correspond 
to  various  human-like  operations  or  features,  second,  conversion  of  EMG  feature  signals  to  control 
signals  to  input  to  actuators  to  operate  the  prosthetic  hand,  and  third,  design  and  building  of  a 
prosthetic  hand  with  necessary  sensors  to  operate  according  to  the  control  signal  that  corresponds 
to  the  EMG  signal  intended  for  the  particular  identified  hand  operation.  In  this  paper,  a  chrono¬ 
logical  overview  of  the  applications  of  control  theory  to  prosthetic  hand  is  presented.  The  overview 
focuses  on  hard  computing  or  control  techniques  such  as  multi- variable  feedback,  optimal,  nonlinear, 
adaptive  and  robust  and  soft  computing  or  control  techniques  such  as  artificial  intelligence,  neural 
networks,  fuzzy  logic,  genetic  algorithms  and  on  the  fusion  of  hard  and  soft  control  techniques. 
This  overview  is  not  intended  to  be  an  exhaustive  survey  on  this  topic  and  any  omissions  of  other 
works  is  purely  unintentional. 

A  hand  is  considered  as  an  agent  of  human  brain  and  is  the  most  intriguing  and  versatile  ap¬ 
pendage  to  the  human  body.  Over  the  last  several  years,  attempts  have  been  made  to  build  a 
prosthetic  hand  to  replace  a  human  hand  to  fully  simulate  the  various  natural/human-like  opera¬ 
tions  of  moving,  grasping,  lifting,  twisting  and  so  on.  Replicating  the  human  hand  in  all  its  various 
functions  is  still  a  challenging  task  due  to  the  extreme  complexity  of  a  human  hand,  which  is  27 
bones,  controlled  by  about  38  muscles  to  provide  the  hand  with  22  degrees  of  freedom  (DOFs),  and 
incorporates  about  17,000  tactile  units  of  4  different  types  Kandel  and  Schartz  (1985);  Zecca  et  al. 
(2002).  Parallels  between  dextrous  robot  and  human  hands  were  explored  by  examining  sensor 
motor  integration  in  the  design  and  control  of  these  robots  through  bringing  together  experimental 
psychologists,  kinesiologists,  computer  scientists,  and  electrical  and  mechanical  engineers. 

Artificial  hands  have  been  around  for  the  last  several  years  developed  by  various  researchers 
in  the  field  and  some  of  the  prosthetic  devices  developed  are  given  below  (in  chronological  order) 
Zecca  et  al.  (2002,  2006): 

1.  Russian  arm  -  Kobrinski  (1960);  Sherman  (1964);  McKenzie  (1965), 

2.  Waseda  hand  -  Kato  et  al.  (1967), 

3.  Boston  arm1  -  Mann  and  Reimers  (1970), 

4.  UNB  hand  (University  of  New  Brunswick)  -  Rohland  (1975);  Hudgins  (1991);  Light  et  al. 

(2002), 

5.  Hanafusa  hand  -  Hanafusa  and  Asada  (1977), 

6.  Crossley  hand  -  Crossley  and  Umholtz  (1977), 

1The  ” Boston  Arm,”  project  involved  the  Harvard  Medical  School,  Massachusetts  General  Hospital,  the  Liberty 
Mutual  Research  and  Rehabilitation  Centers,  and  M.I.T. 
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7.  Okada  hand  -  Okada  (1982), 

8.  Utah/MIT  hand  (University  of  Utah/Massachusetts  Institute,  of  Technology)  -  Jacobsen 
et  al.  (1982,  1985);  Iversen  et  al.  (2005), 

9.  JPL/Stanford  hand  (Jet  Propulsion  Laboratory/Stanford  University)  -  Salisbury  (1982); 
Venkataraman  and  Djaferis  (1987), 

10.  Minnesota  hand  -  Lian  et  al.  (1983), 

11.  Manus  hand  -  Kyberd  and  Pons  (2003);  Pons  et  al.  (2004), 

12.  Kobayashi  hand  -  Kobayashi  (1985), 

13.  Rovetta  hand  -  Rovetta  (1983), 

14.  UT/RAL  hand  -  Kim  et  al.  (1987), 

15.  Dextrous  gripper  -  Van  Brussel  et  al.  (1989), 

16.  Belgrade/USC  hand  (University  of  Belgrade/University  of  Southern  California)-  Bekey  et  al. 
(1990), 

17.  Southampton  hand  (University  of  Southampton,  Southampton,  UK)  -  Kyberd  and  Chappell 
(1994), 

18.  MARCUS  hand  (Manipulation  And  Reaction  Control  under  User  Supervision)  -  Kyberd  et  al. 
(1995), 

19.  Kobe  hand  (Kobe  University,  Japan)  -  Okuno  et  al.  (1996), 

20.  Robonaut  hand  (NASA  Johnson  Space  Center)  -  Lovchik  and  Diftler  (1999), 

21.  NTU  hand  (National  Taiwan  University)  -  Huang  and  Chen  (1999), 

22.  Hokkaido  hand  -  Nishikawa  et  al.  (2001), 

23.  DLR  hand  (Deutschen  Zentrums  fiir  Luft-  und  Raumfahrt-German  Aerospace  Center)  -  Liu 
et  al.  (1999);  Butterfass  et  al.  (2001), 

24.  TUAT/Karlsruhe  hand  (Tokyo  University  of  Agriculture  and  Technology/ University  of  Karl¬ 
sruhe)  -  Fukaya  et  al.  (1996), 

25.  BUAA  hand  (Beijing  University  of  Aeronautics  and  Astronautics)  -  Zhang  et  al.  (2001), 

26.  TBM  hand  (Toronto/Bloorview  MacMillan)  -  Dechev  et  al.  (2001), 

27.  ULRG  System  (University  of  Louisiana  Robotic  Gripper)  -  Kolluru  et  al.  (2002), 

28.  Oxford  hand  -  Kyberd  and  Pons  (2003), 

29.  IOWA  hand  (University  of  Iowa)  -  Yang  et  al.  (2004), 

30.  MA-I  hand  -  Suarez  and  Grosch  (2004), 
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31.  RCH  -1  (ROBO  CASA  hand  1  2)  -  Roccella  et  al.  (2004), 

32.  UB  hand  (University  of  Bologna)  -  Lotti  et  al.  (2005), 

33.  Ottoback  SUVA  hand  -  (www.ottobock.co.uk), 

34.  Northwestern  University  system  -  Farrell  et  al.  (2005), 

35.  SKKU  Hand  II  (Sungkyunkwan  University,  Korea)  -  Choi  et  al.  (2006), 

36.  Applied  Physics  Laboratory  (APL)  at  Johns  Hopkins  University  (JHU)  -  DARPA06b  (2006); 
APL-JHU-06  (2006);  APL-JHU-07  (2007), 

and  some  of  the  commercial  web  sites  for  prosthetic  devices  are 

1.  Sensor  Hand™  Speed  from  Otto  Bock  (www.ottobock.co.uk), 

2.  VASI  (Variety  Ability  Systems  Inc.),  a  company  of  the 
Otto  Boch  Group  (http://www.vasi.on.ca/index.html), 

3.  Utah  Arm  from  Motion  Control  (www.utaharm.com), 

4.  The  i-LIMB  Hand  from  Touch  Bionics  (www.touchbionics.com),  and 

5.  so  on. 

A  very  useful  comparison  table  between  several  hands  listed  above  and  human  hand,  adapted  from 
Zecca  et  al.  (2002,  2006),  is  updated  and  shown  in  Table  1. 

However,  about  35%  of  the  amputees  do  not  use  their  prosthetic  hand  regularly  according  to 
Atkins  et  al.  (1996)  due  to  various  reasons  such  as  poor  functionality  of  the  presently  available 
prosthetic  hands  and  psychological  problems.  To  overcome  this  problem,  one  has  to  design  and  de¬ 
velop  an  artificial  hand  which  “mimics  the  human  hand  as  closely  as  possible”  both  in  functionality 
and  appearance. 

There  are  a  number  of  surveys,  and/or  state-of-the  art  articles  that  appeared  over  the  years  on 
the  subject  of  myoelectric  prosthetic  hand  including  the  work  in  USSR  (Russian)  given  by  Sherman 
(1964)  and  some  of  them  are  given  by  Childress  (1980);  Scott  and  Parker  (1988);  Grupen  et  al. 
(1989);  Sears  and  Shaperman  (1991);  Shimoga  (1996);  Bicchi  (2000);  Okamura  et  al.  (2000);  Zecca 
et  al.  (2002);  Kyberd  et  al.  (2003);  Muzumdar  (2004);  Cotton  et  al.  (2006);  Arimoto  (2008);  Birglen 
et  al.  (2008);  Inoue  and  Hirai  (2009). 

The  complete  details  are  given  in  our  book  chapter  attached  as  Appendix  2A-1. 


Modeling  of  a  Five-Fingered  Smart  Prosthetic  Hand 

Human  Hand  Anatomy 

Figure  2  (a)  shows  a  normal  human  hand  composed  of  thumb  (£),  index  ( i ),  middle  (m),  ring  (r), 
little  (/)  fingers  and  palm.  The  wrist  is  located  between  the  forearm  and  the  hand  and  consists  of 
eight  carpal  bones  organized  in  two  rows  of  proximal  (movable)  and  distal  (immovable)  carpal  bones 
as  shown  in  Figure  2  (b)  Seeley  et  al.  (2007).  The  proximal  row  (top)  of  carpal  bones  from  lateral  to 
medial  is  the  Scaphoid ,  Lunate ,  Triquetrum  and  Pisiform ;  the  distal  row  (bottom)  of  carpal  bones 
from  medial  to  lateral  has  the  Hamate ,  Capitate ,  Trapezoid  and  Trapezium.  The  hand  is  composed 

2 the  Italy- Japan  joint  laboratory  for  Research  on  Humanoid  and  Personal  Robotics 
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Table  1:  comparison  of  Human  Hand  with  Artificial  Hands:  Robotic  and  Prosthetic 
Hands:  Force  indicates  power  grasp  Speed  indicates  the  time  required  for  a  full  closing 
and  opening;  E :  stands  for  external,  I:  stands  for  internal 


Size 

(Norm) 

#  of 
Fingers 

#  of 
DOFs 

#  of 
Sensors 

#  of 

Actuators 

Weight 

(gms) 

Force 

(N) 

Speed 

(sec) 

Controls 

Human  hand 

1.0 

5 

22 

«  17,000 

38(I+E) 

«  400 

>300 

0.25 

E 

Russian  arm 

5 

3 

1 

«  147 

Waseda  hand 

UNB  hand 

Hanafusa  hand 

Crossley  hand 

Utah/MIT  hand 

«  2.0 

4 

16 

16 

32(E) 

- 

31.8 

- 

E 

JPL/Stanford  hand 

«  1.2 

3 

9 

- 

12(E) 

1100 

45 

- 

E 

Minnesota  hand 

Kobayashi  hand 

Rovetta  hand 

UT/RAL  hand 

Dextrous  gripper 

Belgrade/USC  hand 

«1.1 

4 

4 

23+4 

4(E) 

- 

- 

- 

E 

Southampton  hand 

«  1.0 

5 

6 

- 

6(E) 

400 

38 

«  5 

E 

MARCUS  hand 

«  1.1 

3 

2 

3 

2(1) 

- 

- 

- 

I 

Kobe  hand 

Robonaut  hand 

«  1.5 

5 

12+2 

43+ 

14(E) 

- 

- 

- 

E 

NTU  hand 

«  1 

5 

17 

35 

17(E) 

1570 

- 

- 

E 

Hokkaido  hand 

«  1 

5 

17 

35 

17(E) 

1570 

- 

- 

E 

DLR  hand  II 

>1 

5 

7 

- 

7(E) 

125 

- 

- 

E 

TUAT/Karlsruhe  hand 

«  1 

5 

17 

- 

17(E) 

«  120 

12 

0.1 

E 

BUAA  hand 

4 

2 

TBM  hand 

Oxford  hand 

IOWA  hand 

MA-I  hand 

Robo  Casa  hand-1 

«  1 

5 

16 

24 

6(E+I) 

350 

«  40 

0.25 

E 

Ottobock  SUVA 

1 

3 

1 

2 

1(E) 

600 

- 

- 

I 

UB  hand 

Hokkaido  hand 

>1 

5 

7 

- 

7(E) 

125 

- 

- 

E 

Northwestern 

SKKU  Hand  II 

1.1 

4 

4 

- 

3 

900 

APL-JHU  System 
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(a)  Physical  Appearanee  of  Right  Hand 
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(b)  Hones  of  Left  Hand  (Posterior  View) 


Figure  2:  Human  Wrist  and  Hand:  (a)  Physical  Appearance  of  Right  Hand  (Anterior  View)  (b) 
Bones  of  Left  Hand  (Posterior  View) 


of  five  Metacarpals  and  five  Digits.  The  metacarpals  produce  a  curve,  so  the  palm  is  concave  in 
the  resting  position.  The  five  digits  contain  one  thumb  (t)  and  four  fingers,  e.g.  index  (z),  middle 
(m),  ring  (r),  and  little  (/)  fingers,  respectively.  The  thumb  has  two  bones,  Proximal  phalanx 
and  Distal  phalanx.  Each  finger  consists  of  three  bones,  Proximal  phalanx ,  Middle  phalanx  and 
Distal  phalanx.  In  this  work,  we  assumed  that  the  palm  is  fixed,  the  thumb  has  two  links  (proximal 
phalanx  and  distal  phalanx),  and  each  finger  has  three  links  (proximal  phalanx,  middle  phalanx 
and  distal  phalanx). 

Synovial  joints  are  formed  at  the  surface  of  relative  motion  between  two  bones.  The  joints  of 
thumb  and  four  fingers  contain  two  saddle-shaped  articulating  surfaces  between  two  bones  and  can 
be  classified  as  saddle  joints.  Index,  middle,  ring,  and  little  fingers  include  three  revolute  joints  in 
order  to  do  the  angular  movements  (Figure  2  (b)).  Metacarpal-phalangeal  (MCP)  joint  is  located 
between  metacarpal  and  proximal  phalange  bones;  proximal  and  distal  interphalangeal  (PIP  and 
DIP)  joints  separate  the  phalangeal  bones.  Thumb  contains  metacarpal-phalangeal  (MCP)  and 
interphalangeal  (IP)  joints  Seeley  et  al.  (2007).  In  this  work,  qJ2  and  q3  (j  =  i,  m,  r  and  l) 
represent  the  angular  positions  (or  joint  angles)  of  the  first  joint  MCPi ,  the  second  joint  PIPi  and 
the  third  joint  DIPi  of  index,  middle,  ring  and  little  fingers,  respectively;  q\  and  q\  are  the  angular 
positions  of  the  first  joint  MCP 1  and  the  second  joint  IP 1  of  thumb  (£),  respectively. 
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Trajectory  Planning 

The  trajectory  planning  using  cubic  polynomial  was  discussed  in  our  previous  work  Chen  (2009); 
Chen  et  al.  (2008a, b,  2009a, c,  2010)  for  a  two-fingered  (thumb  and  index  finger)  smart  prosthetic 
hand.  For  three-link  fingers,  we  present  the  same  technique  for  fingertip  orientation  control.  A 
time  history  of  desired  (d)  fingertip  orientation  (</>)  and  its  differentiation  (</>  and  </>)  is  given  as 


<t>d(t) 

—  UJq  +  LO\t  +  T  <^3^5 

(i) 

fait) 

—  T  2jUJ2t  T  3ou3t2, 

(2) 

«(*) 

=  2  (jj2  +  6  u3t, 

(3) 

where  cjo-^3  are  undetermined  constants  and  the  superscript  j  indicates  the  index  of  each  finger 
(j  =  i,  m,  r  and  /).  The  relations  (1)  and  (2)  need  to  satisfy  the  constraint  conditions  at  initial 
time  to  and  final  time  tf.  This  can  be  written  as 


T  ft  = 

Here,  the  matrices  T,  fi,  and  <l>  are 


'  i 

to 

t2 

4 

T  = 

0 

i 

1 

tf 

2*o 

t2 

lf 

3  4 

tf 

_  0 

1 

2  tf 

3  tf 

n  =  [ 

LOq 

U 1 

(Z>2 

UJ3 

$  = 

'  K 

</>0 

<4 

Therefore,  the  4  unknown  constants,  cjq-^3?  can  be  computed  by  ft  —  T  x<l>. 


(4) 

(5) 

(6) 
(7) 


Kinematics 

Kinematics  is  the  study  of  geometry  in  motion  and  is  restricted  to  a  natural  geometrical  description 
of  motion  by  the  manners,  including  positions,  orientations,  and  their  derivatives  (velocities  and 
accelerations)  Jazar  (2007);  Siciliano  et  al.  (2009).  Forward  and  inverse  kinematics  of  articulated 
systems  study  the  analytical  relationship  between  the  angular  positions  of  joints  and  the  positions 
and  orientations  of  the  fingertips.  A  desired  trajectory  is  usually  specified  in  Cartesian  space  and 
the  trajectory  controller  is  easily  performed  in  the  joint  space.  Therefore,  conversion  of  Cartesian 
trajectory  planning  to  the  joint  space  Siciliano  et  al.  (2009)  is  necessary.  Using  inverse  kinematics, 
the  joint  angular  positions  of  each  finger  need  to  be  obtained  from  the  known  fingertip  positions. 
Then  the  angular  velocities  and  angular  accelerations  of  joints  can  be  obtained  from  the  linear 
and  angular  velocities  and  accelerations  of  fingertips  by  differential  kinematics.  The  inverse  and 
differential  kinematics  of  two-link  thumb  and  three-link  fingers  were  discussed  in  our  previous 
publications  Chen  (2009);  Chen  et  al.  (2008a, b,  2009a, c,  2010)  for  a  two-fingered  (thumb  and  index 
finger)  smart  prosthetic  hand. 

For  five  fingers  shown  in  Figure  3,  XG,  YG ,  and  ZG  are  the  three  axes  of  global  coordinate. 
Local  coordinate  xt-yt-zt  of  thumb  can  be  reached  by  rotating  through  angles  a  and  (3  to  XG  and  YG 
of  the  global  coordinate,  subsequently.  Local  coordinate  x'l-yl-zl  of  index  finger  can  be  obtained  by 
rotating  through  angle  a  to  XG  and  then  translating  a  vector  d1  of  the  global  coordinate;  similarly, 
the  local  coordinate  x^-yi-z^  of  middle  finger  (j  =  m),  ring  finger  ( j  =  r),  and  little  finger  (j  =  /) 
can  be  obtained  by  rotating  through  angle  a  to  XG  and  then  translating  the  vector  (Y  ( j  =  m,  r 
and  /)  of  the  global  coordinate. 
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Figure  3:  Relationship  between  Global  Coordinate  and  Local  Coordinates:  Local  coordinate  x1-!/1- 
zl  of  thumb  can  be  reached  by  rotating  through  angles  a  and  (3  to  XG  and  YG  of  global  coordinate, 
subsequently.  Local  coordinate  xl-yl-zl  of  index  finger  can  be  obtained  by  rotating  through  angle 
a  to  XG  and  then  translating  a  vector  d1  of  the  global  coordinate. 

Adaptive  Neuro-Fuzzy  Inference  System  (ANFIS) 

The  inverse  kinematics  problems  can  be  solved  by  using  adaptive  neuro-fuzzy  inference  system 
(ANFIS)  method  Jang  et  al.  (1997)  where  the  input  of  fuzzy-neuro  system  is  the  Cartesian  space  and 
the  output  is  the  joint  space.  ANFIS  is  a  fuzzy  inference  system  implemented  in  the  framework  of 
adaptive  networks  which  provides  the  best  optimization  algorithm  to  find  parameters  in  order  to  fit 
the  data.  ANFIS  includes  premise  parameters ,  that  defines  membership  functions,  and  consequent 
parameters ,  that  defines  the  coefficients  of  each  output  equation.  ANFIS  tunes  the  membership 
function  and  identifies  the  coefficients  by  the  backpropagation  gradient  descent  and  least-squares 
methods,  respectively.  ANFIS  uses  a  hybrid  learning  algorithm  to  identify  parameters  of  Sugeno- 
type  fuzzy  inference  systems  for  fuzzy  modeling  procedures  to  learn  information  about  a  data  set 
in  order  to  compute  the  membership  function  and  track  the  given  input/output  data.  Figure  4 
(a)  shows  a  two  input  first-order  Sugeno  fuzzy  model  with  two  rules  and  Figure  4  (b)  depicts 
the  equivalent  ANFIS  structure  for  all  the  computation  below.  Sugeno-type  fuzzy  system  has 
the  following  Rule  Base  Jang  et  al.  (1997);  Karray  and  De  Silva  (2004);  Nguyen  et  al.  (2003); 
Nandikolla  and  Naidu  (2005);  Nandikolla  (2005). 

If  x  is  Ai  and  y  is  £>i,  then  fi  =  p\x  +  q\y  +  r\. 

If  x  is  A2  and  y  is  B 2,  then  /2  =  P2X  +  q2y  +  r2 . 

Here,  x  and  y  are  inputs  to  constitute  the  premise  parameters  A 1,  A2,  Hi,  and  B2  (Layer  1  in 
Figure  4  (b)).  pi ,  and  ri  (i  =  1,2)  are  the  consequent  parameters.  We  evaluate  the  rules  by 
choosing  product  for  T-Norm  (Layers  2  and  3)  which  results  in 

Wi  =  /J,Ai(x)  UBiiy),  i  =  1,2.  (8) 

Here,  yAi(x)  and  //£>, (y)  are  designed  fuzzy  membership  functions.  Now  after  leaving  the  arguments 
(Layer  4),  we  get  the  output  f(x,y)  by  Rule  Consequences. 


f(x,y) 


Wi  (x,  y)fi(x,  y)  +  w2(x,  y)h{x,  y) 
w\  (x,y)  +  w2  (x,y) 


(9) 
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f=PiX  +  q]y  +  r1 

nI' 

f=  Wif  +  WJi 
J  Wl+w2 
=  wjx+  wj2 

t 

f2=p2x  +  q2y  +  r2 


(a)  A  Two  Input  First-OrderSugeno  Fuzzy  Model  with  Two  Rules 

Layer  1  Layer  2  Layer  3  Layer  4  Layer  5 


Figure  4:  ANFIS  Architecture:  (a)  A  Two  Input  First-Order  Sugeno  Fuzzy  Model  with  Two  Rules 
(b)  Equivalent  ANFIS  Structure  Jang  et  al.  (1997).  The  inverse  kinematics  problems  are  solved 
by  using  adaptive  neuro-fuzzy  inference  system  (ANFIS)  method  where  the  input  of  fuzzy-neuro 
system  is  the  Cartesian  space  and  the  output  is  the  joint  space.  ANFIS  is  a  fuzzy  inference  system 
implemented  in  the  framework  of  adaptive  networks  which  provides  the  best  optimization  algorithm 
to  find  parameters  in  order  to  fit  the  data. 


/  (Layer  5)  can  be  written  as 


where 


W\ 


f 


mh  +  W2/2 

Wl  +  IU2 

w  1  ,/'l  +w2f2, 


W 1 

- j - ,  ^2 

W 1  +  W2 


W  2 

Wi  +  W2  ’ 


(10) 

(11) 


Dynamics  of  Hand 


The  dynamic  equations  of  hand  motion  are  derived  via  Lagrangian  approach  using  kinetic  energy 
and  potential  energy  as  Lewis  et  al.  (2004);  Kelly  et  al.  (2005);  Jazar  (2007);  Siciliano  et  al.  (2009); 
Chen  (2009) 

d_  fd£\  _d£_ 

dt  \dq/  <9q  T 

where  C  is  the  Lagrangian;  q  and  q  represent  the  angular  velocities  and  angle  vectors  of  joints, 
respectively;  r  is  the  given  torque  vector  at  joints.  The  Lagrangian  C,  can  be  expressed  as 


C  =  T-V, 


(13) 
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where  T  and  V  denote  kinetic  and  potential  energies,  respectively.  Substituting  (13)  into  (12), 
dynamic  equations  of  thumb  can  be  obtained  as  below. 

M(q)q  +  C(q,  q)  +  G(q)  =  t ,  (14) 

where  M(q)  is  the  inertia  matrix;  C(q,  q)  is  the  Coriolis/centripetal  vector  and  G(q)  is  the  gravity 
vector.  (14)  can  be  also  written  as 

M(q)q  +  N(q,q)  =  r,  (15) 

where  N(q,  q)  =  C(q,  q)  +  G(q)  represents  nonlinear  terms.  The  dynamic  relations  for  the  two-link 
thumb  and  the  remaining  three-link  fingers  are  quite  lengthy  and  omitted  here  due  to  lack  of  space 
Nikoobin  and  Haghighi  (2008);  Arslan  et  al.  (2008);  Chen  (2009). 

Control  Techniques 

Feedback  Linearization 

The  nonlinear  dynamics  represented  by  (15)  is  to  be  converted  into  a  linear  state- variable  system 
using  feedback  linearization  technique  Lewis  et  al.  (2004).  Alternative  state-space  equations  of  the 
dynamics  can  be  obtained  by  defining  the  posit  ion/ velocity  state  x(£)  of  the  joints  as 

x(t)=[q\t)  q '(*)]'•  (16) 

Let  us  repeat  the  dynamical  model  and  rewrite  (15)  as 

|q(t)  =  -M-'fqff))  [N(q(t),  q(())  -  r(t)] .  (17) 

Thus,  from  (16)  and  (17),  we  can  derive  a  linear  system  in  Brunovsky  canonical  form  as 

*(*)  =00  +  I  (18) 

with  its  control  input  vector  given  by 

u (t)  =  —M~1(q(t))  |N(q(t),q(t))  -  r(t)] .  (19) 

Let  us  suppose  the  prosthetic  hand  is  required  to  track  the  desired  trajectory  q d(t)  described  under 
path  generation  or  tracking.  Then,  the  tracking  error  e(t)  is  defined  as 

e(i)  =  qd(t)  ~q(t).  (20) 

Here,  q d(t)  is  the  desired  angle  vector  of  joints  and  can  be  obtained  by  trajectory  planning  Chen 
(2009);  Chen  et  al.  (2008a, b,  2009a, c);  q (t)  is  the  actual  angle  vector  of  joints.  Differentiating  (20) 
twice,  to  get 

e(i)  =  q d(t)  -  q (t),  e(t)  =  q d(t)  -  q(t).  (21) 

Substituting  (17)  into  (21)  yields 

e(i)  =  q d(t)  +  M_1(q (t))  [N(q(t),  q(t))  -  r(t)]  (22) 
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from  which  the  control  function  can  be  defined  as 

U (t)  =  q d(t)  +  M_1(q(t))  [N(q(t),  q(t))  -  r(t)] .  (23) 

This  is  often  called  the  feedback  linearization  control  law,  which  can  also  be  inverted  to  express  it 
as 


r{t)  =  M(q(t))  [qd(t)  -  u(t))  +  N(q(t),  q(t)] . 


(24) 


Using  the  relations  (21)  and  (23),  and  defining  state  vector  x(£)  =  [e'ft)  e'(t)]f  the  tracking  error 
dynamics  can  be  written  as 


*(*)  = 


0  I 
0  0 


x(t)  + 


0 

I 


u  (t). 


Note  that  this  is  in  the  form  of  a  linear  system  such  as 

x(£)  =  Ax(f)  +  Bu(t). 


(25) 


(26) 


Adaptive  Control 

The  tracking  error  e(i)  and  the  hltered  tracking  error  r(i)  are  dehned  as 

e(t)  =  q<j(t)  -q(t),  (27) 

r(t)  =  eft)  +  A  e(£).  (28) 

Here,  q^(t)  is  the  desired  angle  vector  of  joints;  q ft)  is  the  actual  angle  vector  of  joints;  A  = 
diag( Ai,  A2, . . Xn)  is  the  positive-definite  diagonal  gain  matrix.  The  filtered  error  (28)  ensures 
stability  of  the  overall  system  so  that  the  tracking  error  (27)  is  bounded.  Figure  5  shows  the 
block  diagram  of  the  adaptive  controller.  Here,  the  filtered  signal  r ft)  is  derived  from  the  tracking 
error  eft)  and  the  trajectory  planner  and  is  fed  to  the  adaptive  controller  of  the  prosthetic  hand. 
Differentiating  and  substituting  (28)  into  (14)  gives  the  dynamic  equation  in  terms  of  the  filtered 
error  rft)  as 

M(q(i))f(t)  =  -CTO(q(t),  q(t))r(t)  +  f(t)  -  r(t),  (29) 

where  C(q (t),  q (t))  =  Cm(q(t),  q(t))q(t)  and  the  nonlinear  term  f  (t)  can  be  defined  as 

f  ft)  =  M(q(t))(qd(i)  +  Ae(t))  +  G(q  ft))  + 

Cm(q(t),  q(t))(qd(t)  +  A  eft))  +  rdis, 

=  Ytt.  (30) 

Here,  is  the  unknown  disturbance.  Y  is  a  regression  matrix  of  known  robot  functions  and  7V 
is  a  vector  of  unknown  parameters  Lewis  et  al.  (1999).  The  regression  matrix  Y  and  the  unknown 
parameter  vector  7r  of  two-link  thumb  and  three-link  index  finger  are  given  in  Appendix  Chen 
(2009).  The  torque  vector  r(t)  can  be  calculated  by 

rft)  =  f  ft)  +  KDr(i).  (31) 

The  unknown  parameter  rate  vector  7r  can  be  updated  by 

7z  =  T^Y'rCO  (32) 

where  T  is  a  tuning  parameter  diagonal  matrix. 
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Figure  5:  Block  Diagram  of  Adaptive  Control  Technique 

Finite-Time  Linear  Quadratic  Optimal  Control 

Figure  6  shows  the  block  diagram  of  finite-time  linear  quadratic  optimal  controller  for  prosthetic 
hand.  For  the  linear  system  (26),  we  can  formulate  the  well-known  finite-time  linear  quadratic 
optimal  control  problem  by  defining  a  performance  index  J  Naidu  (2003)  such  as 

+7:  f  [x'(i)Q(i)x(i)  +  u'(i)R(i)u(i)]  dt  (33) 

2  Jt0 

where  the  terminal  cost  matrix  F  (tf)  and  the  error  weighted  matrix  Q  (t)  are  positive  semidefinite 
matrices,  respectively;  the  control  weighted  matrix  R (t)  is  a  positive  definite  matrix.  The  optimal 
control  u*  (t)  is  given  by 

u  *(t)  =  — R_1(t)B'P(t)x*(t)  =  -K  (t)x*(t).  (34) 

Here,  K (t)  =  R_1(t)B'P(t)  is  called  Kalman  gain  and  P (£),  the  symmetric  positive  definite  matrix 
(for  all  t  G  [to,  tf]),  is  the  solution  of  the  matrix  differential  Riccati  equation  (DRE) 


P (t)  =  — P(t)A  -  A'P(t)  -  Q(t)  +  P(f)BR_1(t)B'P(t) 
satisfying  the  final  condition 


P(t  =  t/)  =  F(t/). 


(35) 

(36) 
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Figure  6:  Block  Diagram  of  Hybrid  Optimal  Controller  for  Prosthetic  Hand 


The  optimal  state  x*  is  the  solution  of 

x*(t)  =  [A  -  BR_1(t)B'P(t)]  x*(t).  (37) 

Therefore,  the  required  torque  r*(£)  can  be  calculated  by  the  optimal  control  u *(£). 

r* (t)  =  M(q (t) )  (4i(t)  -  u* (t))  +  N (q(t) ,  q(i) ) .  (38) 

The  Stability  of  Finite-Time  Optimal  Control 

Our  previous  work  Chen  (2009);  Chen  et  al.  (2009b, c)  shows  optimal  control  can  avoid  overshooting 
and  get  better  results  than  PID  control,  but  the  regular  optimal  control  takes  time  to  be  convergent. 
To  study  the  convergent  stability  of  the  optimal  control,  we  modify  the  performance  index  J  Naidu 
(2003)  as 


J  =  + 

\  f  e2at  [x'(t)Q(t)x(t)  +  u'(t)R(t)u(t)]  dt,  (39) 

2  Jt0 

where  a  is  a  positive  parameter.  To  find  the  optimal  control  which  minimizes  the  performance  index 
(39)  under  the  dynamical  constraint  (26).  This  problem  can  be  solved  by  changing  the  previous 
system,  so  the  following  transformations  can  be  developed  as 

x(t)  =  eatx(£) ;  u(t)  =  eatu(t) .  (40) 
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Then,  using  the  transformations  (40),  it  is  easy  to  see  that  the  modified  system  becomes 

x(i)  =  -^{eatx(£)}  =  aeatx(t)  +  eatx(£) 

=  Q'x(t)  +  ed[Ax(t)  +  Bu(t)] 
x(t)  =  (A  +  Q'I)x(t)  +  Bu(t).  (41) 

Considering  the  minimization  of  the  modified  system  (41)  and  the  performance  index  (39),  we 
see  that  the  optimal  control  Naidu  (2003)  is  given  by 

u  *(t)  =  — R_1B'Px*(t)  =  -Kx*(t),  (42) 

where  K  =  R_1B'P  and  the  matrix  P  is  the  positive  definite,  symmetric  solution  of  the  algebraic 
Riccati  equation 


P(A  +  al)  +  (A'  +  aI)P  -  PBR^B'P  +  Q  =  0.  (43) 

Using  the  optimal  control  (42)  in  the  modified  system  (41),  we  get  the  optimal  closed-loop  system 
as 


i*(t)  =  (A +  al  —  BR_1B'P)x*(t).  (44) 

Hence,  applying  the  transformations  (41)  in  the  modified  system  (42),  the  optimal  control  of  the 
original  system  (26)  and  the  associated  performance  measure  (39)  is  given  by 

u  *(t)  =  e~atu  *(t)  =  -e^R^B'P  eatx*(t) 

=  — R_1B'Px*(t)  =  — Kx*(£).  (45) 

We  see  that  the  closed-loop  optimal  control  system  (44)  has  eigenvalues  with  real  parts  less  than 
—a.  In  other  words,  the  state  x*(£)  approaches  zero  at  least  as  fast  as  e~at. 

GA-Based  PID  Hybrid  Control 

Figure  7  shows  the  block  diagram  of  a  hybrid  GA-based  PID  controller  for  the  presented  five¬ 
fingered  prosthetic  hand  with  control  signal  as 

u (t)  =  -Kpe(t)  —  Ki  J  e(t)dt  —  Koe(f)  (46) 

with  the  proportional  Kp,  integral  Kj,  and  derivative  Kp  diagonal  gain  matrices.  We  then  rewrite 
(24)  as 


r{t)  =  M(q(t))[qd(t)  +  KPe(i)  +  Kx  J  e(t)dt 

+KDe(t))+N(q(t),q(t)]-  (47) 

Then  we  use  GA  to  tune  all  gain  coefficients  Kp,  Kd  and  Kp  of  PID  controller.  Figure  8 
shows  the  flowchart  of  GA  and  the  procedure  is  briefly  stated  below. 

1.  Define  the  GA  parameters:  include  initial  population,  population  at  the  end  of  the  first 
generation,  number  of  chromosomes  kept  for  mating,  mutation  rate,  and  tolerance  e  so  on. 
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Figure  7:  Block  Diagram  of  the  Hybrid  GA-Based  PID  Controller  for  14-DOF  Five-Fingered  Pros¬ 
thetic  Hand 

2.  Create  a  homogeneous  population:  generate  N  elements  (chromosomes)  and  N  is  the  initial 
population. 

3.  Evaluate  cost  (fitness)  function  of  each  chromosome:  calculate  the  fitness  value  of  the  zth 
member  in  the  population. 

4.  Select  mate  based  on  the  performance  of  each  gene:  create  a  new  population  from  the  current 
population  based  on  the  ranking  of  the  current  fitness  value,  e.g.  determine  which  parents 
participate  in  producing  offspring  for  the  next  generation. 

5.  Reproduce  the  generation  by  crossover:  use  the  single  or  multiple  crossover  points  to  generate 
new  chromosomes  that  retain  the  good  feature  and  discard  the  bad  feature. 

6.  Mutate:  utilize  the  mutation  rate  which  can  randomly  mutate  the  gene  to  avoid  falling  into 
the  local  minima  area. 

7.  Repeat  steps  3  to  6  until  it  reaches  the  maximum  number  of  iterations  or  stopping  condition 
defined  by  e  is  satisfied. 
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Figure  8:  The  Flowchart  of  Genetic  Algorithm  (GA) 

Fuzzy  Logic-Based  PD  Hybrid  Control 

Figure  9  shows  the  block  diagram  of  the  hybrid  fuzzy  logic-based  PD  controller  for  the  presented 
five-fingered  prosthetic  hand  with  control  signal  as 

u (t)  =  -KP(t)e(t)  -  KD(t)e(t)  (48) 

with  the  proportional  Kp(f)  and  derivative  Kd  (£)  diagonal  gain  matrices  with  time  varying.  We 
then  rewrite  (24)  as 


r{t)  =  M(q(t))[qd(t)  +  KP(t)e(t)+KD(t)e(t))  + 

N(q(t)»q(*)]-  (49) 

Then  we  use  mamdani  fuzzy  inference  system  to  tune  the  time- varying  parameters  Kp(f)  and 
Kd  (£)  of  PD  controller.  Figure  10  shows  the  structural  characteristics  of  proposed  fuzzy  inference 
system,  which  includes  two  inputs  (error  e  and  error  change  e)  on  the  left  and  one  output  (Kp) 
on  the  right.  Each  input  or  output  layer  contains  seven  triangular  membership  functions  as  shown 
in  Figure  11  and  49  logic  rules  as  listed  in  Table  2.  After  using  49  logic  rules,  the  output  surface 
Kp  of  fuzzy  inference  system  is  generated  as  shown  in  Figure  12.  Similarly,  Kd  can  be  computed 
by  the  same  way. 
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Figure  9:  Block  Diagram  of  the  Presented  Hybrid  Fuzzy  Logic-Based  Proportional-Derivative  (PD) 
Controller  for  a  Five-Fingered  Prosthetic  Hand:  Errors  and  error  changes  are  calculated  by  actual 
and  desired  angles,  which  are  based  on  adaptive  neuro-fuzzy  inference  system  (ANFIS)  trajectory 
planner.  Then  fuzzy  logic  controller  tunes  all  parameters  of  closed-loop  PD  control  so  that  the 
required  torque  of  the  prosthetic  hand  nonlinear  system  is  computed  by  control  input. 
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Figure  10:  The  Structural  Characteristics  of  Proposed  Fuzzy  Inference  System:  Two  inputs  (error 
e  and  error  change  e)  on  the  left  and  one  output  (tuned  Kp)  on  the  right  with  49  logic  rules 
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Figure  11:  All  Membership  Functions:  Each  of  Two  Inputs  (Upper  Panel)  and  One  Output  (Lower 
Panel)  Uses  7  Triangular  Membership  Functions 


Table  2:  A  Complete  Fuzzy  Logic  Rule  Base 


e\e 
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S:  small;  V:  very 


Figure  12:  The  Output  Surface  of  Fuzzy  Inference  System  with  Two  Inputs  and  49  Logic  Rules 
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Simulation  Results  and  Discussion 


Adaptive  Control 

Figure  13  shows  that  five-fingered  prosthetic  hand  with  14  DOFs  is  reaching  a  rectangular  rod  in 


Figure  13:  Five-Finger  Prosthetic  Hand  Reaching  a  Rectangular  Rod 

order  to  grasp  the  object.  When  thumb  and  the  other  four  fingers  are  performing  extension/flexion 
movements,  the  workspace  of  fingertips  is  restricted  to  the  maximum  angles  of  joints.  Referring 
to  inverse  kinematics,  the  first  and  second  joint  angles  of  the  thumb  fingertip  are  constrained  in 
the  ranges  of  [0,90]  and  [-80,0]  (degrees).  The  first,  second,  and  third  joint  angles  of  the  other  four 
fingers  are  constrained  in  the  ranges  of  [0,90],  [0,110]  and  [0,80]  (degrees),  respectively  Lavangie 
and  Norkin  (2001). 

Next,  we  present  simulations  with  an  adaptive  controller  for  the  14  DOFs  five- fingered  smart 
prosthetic  hand.  The  parameters  of  the  two-link  thumb/three-link  fingers  Arimoto  (2008)  were 
related  to  desired  trajectory.  All  parameters  of  the  smart  prosthetic  hand  selected  for  the  simula¬ 
tions  are  given  in  Table  3  and  the  side  length  and  length  of  the  target  rectangular  rod  are  0.010 
and  0.100  (m),  respectively.  All  initial  actual  angles  are  zero.  The  relating  parameters  between 
the  global  coordinate  and  the  local  coordinates  are  defined  in  Table  4.  Besides,  in  this  work,  we 
assumed  that  each  link  of  all  fingers  is  a  circular  cylinder  with  the  radius  (R)  0.010  (m),  so  the 
inertia  IJzzk  of  each  link  k  of  all  fingers  j  (=  £,  i,  m,  r  and  l)  can  be  calculated  as 

TLk  =  \miR2  +  \miLk-  (50) 

Figure  14,  Figure  16,  Figure  18,  Figure  20  and  Figure  22  show  the  tracking  errors  of  thumb, 
index,  middle,  ring,  and  little  fingers  for  the  proposed  five-fingered  smart  prosthetic  hand,  respec¬ 
tively.  Figure  15,  Figure  17,  Figure  19,  Figure  21  and  Figure  23  show  the  desired/actual  angles  of 
thumb,  index,  middle,  ring,  and  little  fingers  for  the  proposed  five-fingered  smart  prosthetic  hand, 
respectively.  The  observation  that  all  tracking  errors  dramatically  drop  within  one  second  and  are 
less  than  one  degree  after  convergence  provides  the  evidence  that  the  adaptive  controller  for  the 
14-DOFs  prosthetic  hand  enhances  performance.  The  other  observation  that  after  convergence,  all 
three-link  fingers  show  more  unstable  errors  than  two-link  thumb  suggests  that  the  more  DOFs 
increase  the  difficulty  of  the  adaptive  controller  without  knowing  the  mass  and  inertia  of  the  links 
of  all  fingers. 
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Table  3:  Parameter  Selection  of  the  Smart  Hand 


Parameters 

Values 

Thumb 

Time  (£0,£/)* 

Desired  Initial  Position  (Xq,Yq)** 

Desired  Final  Position 

Desired  Initial  Velocity  (Xq,Yq)* 

Desired  Final  Velocity  (Xj,Yj)* 

Length  (L\,L\) 

Mass  (774,7774) 

0,  20  (sec) 

0.035,  0.060  (m) 

0.0495,  0.060  (m) 

0,  0  (m/s) 

0,  0  (m/s) 

0.040,  0.040  (m) 

0.043,  0.031  (kg) 

Index  Finger 

Desired  Initial  Position  (Xq,Yq)** 

Desired  Final  Position  (X^Y^)** 

Length  (L\,L\,L\) 

Mass  (777^,7772,7773) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 

0.045,  0.025,  0.017  (kg) 

Middle  Finger 

Desired  Initial  Position  (X™,V0m)** 
Desired  Final  Position  (X™,!^71)** 
Length  (Lf  L™) 

Mass  ( 777^, 777^, 777 g2) 

0.065,  0.080  (m) 

0.005,  0.060  (m) 

0.044,  0.044,  0.033  (m) 

0.050,  0.028,  0.017  (kg) 

Ring  Finger 

Desired  Initial  Position  (Xq ,V0r)** 

Desired  Final  Position  (X£,YJ)** 

Length  (L\,Lr2,Lrz) 

Mass  (774,772,2,7773) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 

0.041,  0.023,  0.014  (kg) 

Little  Finger 

Desired  Initial  Position  (Xq,Yq)** 

Desired  Final  Position  (Xj.,Vj)** 

Length  (L[,Ll2,Ll3) 

Mass  (jn\,rnl2,rnl3) 

0.055,  0.080  (m) 

0.020,  0.060  (m) 

0.036,  0.036,  0.027  (m) 

0.041,  0.023,  0.014  (kg) 

*  All  fingers  use  same  parameters 

** All  parameters  are  in  local  coordinates 


Table  4:  Parameter  Selection  of  Conversion  between  Global  and  Local  Coordinates 


Parameters 

Values 

Rotating  a 

90  (deg) 

Rotating  (3 

45  (deg) 

Translating  d1 

(0.035,  0,  0)  (m) 

Translating  dm 

(0.040,  0,  -0.020)  (m) 

Translating  dr 

(0.035,  0,  -0.040)  (m) 

Translating  d1 

(0.025,  0,  -0.060)  (m) 
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Figure  14:  Tracking  Errors  of  Adaptive  Controller  for  Two-Link  Thumb 
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Figure  15:  Tracking  Angles  of  Adaptive  Controller  for  Two-Link  Thumb 


Figure  16:  Tracking  Errors  of  Adaptive  Controller  for  Three-Link  Index  Finger 


Page  76  of  531 


Figure  17:  Tracking  Angles  of  Adaptive  Controller  for  Three-Link  Index  Finger 


Figure  18:  Tracking  Errors  of  Adaptive  Controller  for  Three-Link  Middle  Finger 
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Figure  19:  Tracking  Angles  of  Adaptive  Controller  for  Three-Link  Middle  Finger 


Optimal  Control 

Referring  to  inverse  kinematics,  Figure  24  exhibits  that  the  fingertip  workspace  of  the  thumb  and 
index  finger.  We  present  simulations  with  the  stability  of  optimal  controller  for  the  two-link  thumb 
and  three-link  index  finger  of  a  smart  prosthetic  hand.  All  used  parameters  are  the  same  to  our 
previous  work  Chen  et  al.  (2009c).  The  initial  and  final  orientation  </>o  and  (j)f  are  75  and  160 
degrees,  respectively.  Figure  25  displays  the  tracking  errors  and  desired/actual  angles  of  thumb  for 
the  smart  prosthetic  hand  by  using  the  presented  optimal  controller  with  the  stability  (a  =  0,  0.1, 
and  1).  Figure  26  to  Figure  28  are  the  tracking  errors  and  desired/actual  angles  of  index  finger  for 
the  smart  prosthetic  hand  (a  =  0,  1,  and  10).  To  compare  with  the  original  system  (a  =  0),  these 
results  demonstrate  that  the  presented  optimal  controller  behaves  fast  convergent  and  accurate  as 
the  stability  a  increases.  For  the  three-link  index  finger,  we  design  the  orientation  to  calculate  the 
angular  positions  via  inverse  kinematics.  To  compare  with  our  previous  works  Chen  (2009);  Chen 
et  al.  (2009b, c, a)  by  using  adaptive  neuro-fuzzy  inference  system  (ANFIS)  and  genetic  algorithm 
(GA),  the  simulations  show  that  the  presented  method  is  not  time  consuming. 
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Figure  20:  Tracking  Errors  of  Adaptive  Controller  for  Three-Link  Ring  Finger 


Figure  21:  Tracking  Angles  of  Adaptive  Controller  for  Three-Link  Ring  Finger 


Page  79  of  531 


45 

40 

35 

30 

oti 

O 

Joint  1  of  Little  Finger 

3  25 

1  - Joint  2  of  Little  Finger 

52 

O 

Joint  3  of  Little  Finger 

!§  20 

■3  15 

(73 

H 

10 

5 

n 

-5 

30  5  10  15  20 

Time  (sec) 


Figure  22:  Tracking  Errors  of  Adaptive  Controller  for  Three-Link  Little  Finger 


Figure  23:  Tracking  Angles  of  Adaptive  Controller  for  Three-Link  Little  Finger 
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Figure  24:  The  Fingertip  Workspace  of  Thumb  and  Index  Finger  and  a  Square  Object 


Figure  25:  Tracking  Errors  (left)  and  Joint  Angles  (right)  of  Thumb  for  Optimal  Control  with  the 
Stability  (a  =  0,  0.1,  and  1) 
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Figure  26:  Tracking  Errors  (left)  and  Joint  1  Angles  q\  (right)  of  Index  Finger  for  Optimal  Control 
with  the  Stability  (a  =  0,  1,  and  10) 


Figure  27:  Tracking  Errors  (left)  and  Joint  2  Angles  q\  (right)  of  Index  Finger  for  Optimal  Control 
with  the  Stability  (a  =  0,  1,  and  10) 
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Figure  28:  Tracking  Errors  (left)  and  Joint  3  Angles  (right)  of  Index  Finger  for  Optimal  Control 
with  the  Stability  (a  =  0,  1,  and  10) 
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GA-Based  PID  Hybrid  Control 

We  present  simulations  with  a  PID  controller  and  GA-tuned  PID  controller  for  the  14  DOFs  five¬ 
fingered  smart  prosthetic  hand  grasping  a  rectangular  object  as  shown  in  Figure  13.  All  parameters 
of  the  smart  prosthetic  hand  selected  for  the  simulations  are  the  same  as  our  previous  works  Chen 
et  al.  (2009b,  2010).  All  initial  actual  angles  are  zero  and  the  diagonal  coefficients,  Kp,  Kj  and 
Kd,  for  the  PID  controller  alone  are  arbitrarily  chosen  as  100.  From  the  derived  dynamic  and 
control  models,  after  the  parameters  (Kp,  Ki  and  Kd)  are  determined,  the  torque  matrix  r  can 
be  computed,  and  then  the  squared-tracking  errors  ej(t)  of  the  joint  i  of  the  finger  j  are  obtained. 
Therefore,  the  total  error  E{t),  which  is  a  time-dependent  function,  can  be  described  as 

E(t)=  [  (el(t))2dt,  (51) 

where  to  and  tf  are  initial  and  terminal  time,  respectively.  The  tuned  diagonal  parameters  (Kp, 
Ki  and  Kd)  and  the  total  error  E(t)  of  PID  controller  by  GA  are  listed  in  Table  5.  To  study 


Table  5:  Parameter  Selection  of  G A- Tuned  PID  Controller  and  Computed  Total  Errors 


Input  Output 


Fingers 

KP 

Ki 

Kd 

m 

Case  I 

[976,956] 

[779,279] 

[170,236] 

0.3107 

Case  II 

[988,999] 

[  78,848] 

[  80,109] 

0.1557 

Case  III 

[199,198] 

[127,157] 

[104,102] 

0.8100 

Index 

[794,398,960 

[960,918,914] 

[15,59,242 

0.0465 

Middle 

[794,398,960 

[960,918,914] 

[15,59,242: 

0.1003 

Ring 

[794,398,960 

[960,918,914] 

[15,59,242: 

0.0465 

Little 

[794,398,960 

[960,918,914] 

[15,59,242: 

0.0607 

whether  the  tuned  parameter  range  influences  total  tracking  errors,  we  design  three  different  cases 
with  altering  lower  and  upper  bounds  of  tuned  parameter  ranges  for  two-link  thumb.  Cases  I, 
II,  and  III  for  the  thumb  represent  that  the  PID  parameters  Kp,  Ki  and  Kd  are  constricted  in 
three  different  bounded  ranges  [100,1000],  [50,1000],  and  [100,200],  respectively.  Figure  29  and 
Figure  30  show  that  tracking  errors  and  desired/actual  angles  of  joints  1  and  2  of  PID  and  GA- 
based  PID  controllers  for  Thumb.  These  simulations  show  that  the  large  ranges  [100,1000]  (Case  I) 
and  [50,1000]  (Case  II)  provide  better  results  than  the  PID  controller  parameters  arbitrarily  chosen 
as  100.  However,  the  small  range  [100,200]  (Case  III)  gives  worse  result  than  the  PID  controller 
alone.  These  results  suggest  that  the  bigger  parameter  range,  the  smaller  the  total  error.  Cases  I 
and  II  explain  that  GA  finds  some  parameter  values  E  [100,1000]  and  [50,100]  escaping  the  local 
minimum  area.  Case  III  covers  the  value  100  in  lower  bound,  but  both  total  error  and  convergent 
speed  are  even  worse  than  PID  alone,  suggesting  that  GA  performs  better  for  a  large  range,  but  is 
poor  for  searching  on  the  boundary.  To  further  consider  the  convergent  speed,  Case  I  gives  smaller 
total  error,  but  does  not  improve  its  convergent  speed  when  comparing  to  PID  control  alone.  Yet, 
Case  II  gives  good  total  error  and  convergent  speed.  Case  III  gives  poor  total  error  and  convergent 
speed.  Taken  together,  these  results  imply  that  the  global  minimum  could  be  located  in  the  ranges 
[50,100]  and  [200,1000]  and  the  parameter  ranges  play  an  important  role  in  GA  tuning.  Based  on 
these  findings,  we  use  the  range  [50,1000]  for  the  remaining  three-link  fingers.  Figures  31  to  34 
show  the  simulations  of  PID  and  GA-based  PID  controllers  for  the  remaining  three-link  fingers. 
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Figure  29:  Tracking  Errors  of  Joint  1  (left)  and  Joint  2  (right)  of  PID  and  GA-Based  PID  Controllers 
for  Thumb 


Figure  30:  Tracking  Angles  of  Joint  1  (left)  and  Joint  2  (right)  of  PID  and  GA-Based  PID  Con¬ 
trollers  for  Thumb 
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Figure  31:  Tracking  Errors  (left)  and  Joint  Angles  (right)  of  PID  and  GA-Based  PID  Controllers 
for  Index  Finger 


Figure  32:  Tracking  Errors  (left)  and  Joint  Angles  (right)  of  PID  and  GA-Based  PID  Controllers 
for  Middle  Finger 
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Figure  33:  Tracking  Errors  (left)  and  Joint  Angles  (right)  of  PID  and  GA-Based  PID  Controllers 
for  Ring  Finger 


Figure  34:  Tracking  Errors  (left)  and  Joint  Angles  (right)  of  PID  and  GA-Based  PID  Controllers 
for  Little  Finger 
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Fuzzy  Logic-Based  PD  Hybrid  Control 

In  this  study,  we  assumed  that  each  link  of  all  fingers  is  a  circular  cylinder  with  the  radius  (R) 
0.010  (m),  so  the  inertia  IJzzk  of  each  link  k  of  all  fingers  j  (=  £,  i,  m,  r  and  l)  can  be  calculated  as 

Jlzk  =  \mkR 2  +  \mkLk-  (52) 

All  initial  actual  angles  are  zero  and  the  diagonal  coefficients,  Kp,  Kj  and  Kd,  for  the  PD  or  PID 
controller  alone  are  arbitrarily  chosen  as  100.  From  the  derived  dynamic  and  control  models,  after 
the  parameters  (Kp  and  Kd)  are  selected,  the  control  signal  u  and  torque  r  can  be  calculated. 
Figure  35  shows  tracking  errors  (e\  and  e\  on  the  left  column)  and  desired/actual  angles  (q\  and 


s 


5 


- PD  (Kp  =  1 00) 

100 

L 

- PID  (Kp- 100) 

- Fuzzy+PD  (ECp  e  [50,1000]) 

- Fuzzy+PD  (Kp  e  [50,2000]) 

00 

u 

3 

in. 

JU 

50 

0 

V 

f  ----Desired 

- PD  (Kp  =  1 00) 

_ PID  (ECp  =  1 00) 

\  / 
\  / 

Jf 

-50 

- Fuzzy+PD  (Kp  e  [50,1000]) 

-100 

- Fuzzy+PD  (Kp  e  [50,2000]) 

-5°0  5  10  15  20  0  5  10  15  20 

Time  (sec)  Time  (sec) 


i~\ 

/  \ 

; 

0 

J  -5° 

fv  _ , _ _ 

f 

----Desired 

Ut  - PD  (Kp  =  1 00) 

*^-100 

- PD  (ECp  =  100) 

I  - PID  (Kp- 100) 

t n 

u 

_ PID  (ECp  =  100) 

- Fuzzy+PD  (Kp  e  [50,1000])  , 

lib 

|  -150 

- Fuzzy+PD  (Kp  e  [50,1000])  , 

1  - Fuzzy+PD  (ECp  e  [50,2000]) 

-700 

- Fuzzy+PD  (ECp  e  [50,2000]) 

•60* - -  -  -200* - -  - - - 

0  5  10  15  20  0  5  10  15  20 

Time  (sec)  Time  (sec) 


Figure  35:  Tracking  Errors  (Left)  and  Desired/ Actual  Angles  (Right)  of  Joints  1  (Upper)  and  2 
(Lower)  for  Two-Link  Thumb  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic  Based  PD 
(Solid  Line)  Controllers 

q\  on  the  right  column)  of  joints  1  (the  top  row)  and  2  (the  bottom  row)  for  two-link  thumb  using 
PD  (dash  line),  PID  (dot  line)  and  fuzzy  logic  based  PD  (solid  line)  controllers.  The  tracking  errors 
for  PD  control  are  convergent  within  5  seconds  without  overshooting,  but  PID  control  acts  longer 
(approximate  10  seconds)  with  overshooting  and  oscillation.  The  proposed  hybrid  fuzzy  logic  PD 
control  using  parameters  Kp  G  [50, 1000]  and  Kd  G  [50,  500]  provides  faster  convergence  than  both 
PD  and  PID  controllers.  To  further  study  whether  the  parameter  range  influences  tracking  errors, 
we  found  that  the  larger  parameter  range,  the  faster  convergent  speed  after  altering  Kp  G  [50,  2000] 
without  additional  computational  time.  We  also  used  fuzzy  logic  controller  with  two  inputs  (error 
and  error  rate)  and  one  output  (control  signal),  but  the  control  system  could  not  obtain  convergent 
tracking  errors  (data  not  shown).  These  data  suggest  that  fusion  of  soft  computing  (SC)  and  hard 
control  (HC)  is  superior  to  either  HC  or  SC  methodology  alone. 

The  time- variant  computed  control  signals  (u\  and  u^)  and  torques  (r\  and  r|)  for  two-link 
thumb  within  the  first  0.01  second  are  shown  in  Figure  36,  suggesting  that  the  presented  hybrid 
fuzzy  logic  PD  controller  requires  more  power  (torque)  than  PD  and  PID  controllers  in  order  to 
obtain  faster  convergent  tracking  errors. 

Similarly,  Figures  37,  39,  41  and  43  show  tracking  errors  and  desired/actual  angles  for  all 
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Time  (sec)  Time  (sec) 


Figure  36:  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Upper)  and  2  (Lower) 
for  Two-Link  Thumb  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic  Based  PD  (Solid 
Line)  Controllers  in  0.01  Second 


remaining  three-link  fingers.  Figures  38,  40,  42  and  44  show  control  signals  and  torques  for  the 
mapping  three-link  fingers. 
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Figure  37:  Tracking  Errors  (Left)  and  Desired/ Actual  Angles  (Right)  of  Joints  1  (Top),  2  (Middle) 
and  3  (Bottom)  for  Three-Link  Index  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy 
Logic  Based  PD  (Solid  Line)  Controllers 


Figure  38:  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Top),  2  (Middle)  and 
3  (Bottom)  for  Three-Link  Index  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic 
Based  PD  (Solid  Line)  Controllers  in  0.01  Second 
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Figure  39:  Tracking  Errors  (Left)  and  Desired/ Actual  Angles  (Right)  of  Joints  1  (Top),  2  (Middle) 
and  3  (Bottom)  for  Three-Link  Middle  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy 
Logic  Based  PD  (Solid  Line)  Controllers 


Figure  40:  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Top),  2  (Middle)  and 
3  (Bottom)  for  Three-Link  Middle  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic 
Based  PD  (Solid  Line)  Controllers  in  0.01  Second 
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Figure  41:  Tracking  Errors  (Left)  and  Desired/ Actual  Angles  (Right)  of  Joints  1  (Top),  2  (Middle) 
and  3  (Bottom)  for  Three-Link  Ring  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy 
Logic  Based  PD  (Solid  Line)  Controllers 


Figure  42:  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Top),  2  (Middle)  and 
3  (Bottom)  for  Three-Link  Ring  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic 
Based  PD  (Solid  Line)  Controllers  in  0.01  Second 
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Figure  43:  Tracking  Errors  (Left)  and  Desired/ Actual  Angles  (Right)  of  Joints  1  (Top),  2  (Middle) 
and  3  (Bottom)  for  Three-Link  Little  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy 
Logic  Based  PD  (Solid  Line)  Controllers 
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Figure  44:  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Top),  2  (Middle)  and 
3  (Bottom)  for  Three-Link  Little  Finger  Using  PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic 
Based  PD  (Solid  Line)  Controllers  in  0.01  Second 
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1.  Power  grasping  control  for  fully  actuated  prosthetic  hand 


1.1.  Introduction 

Human  hand  is  one  of  the  most  important  and  complex  parts  of  the  body,  which  has  the  ability  to  handle  different 
tasks.  The  ultimate  goal  of  robotic  hand  is  to  achieve  the  functionality  of  a  human  hand.  In  the  past  three  decades, 
there  have  been  numerous  investigations  to  achieve  dexterity  and  ability  of  human  hand,  especially  in  the  fields  of 
humanoid  robotics  and  prosthetic  hand  [1-6].  In  spite  of  all  these  advances  in  this  field,  the  current  state  of  research 
on  prosthetic  hands  is  far  from  that  objective  of  achieving  the  functionality  of  human  hand.  Commercially  available 
prosthetic  hands  have  very  limited  functionality  and  they  are  just  simple  grippers.  The  present  research  on  prosthetic 
hands  involves  complex  control  schemes  to  achieve  the  most  important  functions  of  the  hand  [7-8]. 

Grasping  can  be  categorized  into  two  main  groups:  precision  and  power  grasping.  In  precision  grasping  the  object  is 
held  by  tips  of  the  fingers,  while  in  power  grasping,  the  whole  the  finger  is  active  and  in  contact  with  the  object  [9]. 
Many  research  works  addressed  the  precision  grasping  problem  [12-15]  mainly  because  of  the  well-established 
techniques  for  control  of  end  effector;  however  the  problem  of  power  grasping  is  not  studied  in  depth. 

Many  control  methods  require  the  knowledge  of  the  shape  of  the  object.  For  humans  this  information  is  available  by 
visual  feedback  from  eyes,  while  in  case  of  a  prosthetic  hand  this  visual  information  is  not  directly  available  for 
hand  controller,  and  the  only  available  information  is  electromyographic  (EMG)  signal  related  to  patienfs  arm 
muscle  activities.  However,  normally  the  EMG  signal  is  not  available  for  all  individual  joints  and  besides,  due  to 
measurement  noise,  accessing  high  quality  EMG  signal  is  hard  [10].  Moreover,  using  EMG  signal  to  control  all  the 
movements  requires  lot  of  attention  during  grasping  and  leads  to  fatigue  for  the  amputees  [11].  Hence  it  is  required 
for  prosthetic  hand  to  be  semi-autonomous  which  means  a  part  of  command  information  will  be  provided  by  the 
EMG  signal  and  the  rest  of  the  required  command  should  be  provided  automatically  by  hand  controller. 

Defining  finger  trajectory  without  the  knowledge  of  shape  of  object  to  be  grasped  is  a  challenging  task  for  many 
path  planning  techniques.  For  multi  DOFs  robots  there  are  two  common  methods  for  trajectory  planning  which  are 
inverse  kinematic”  and  — merse  dynamic”  [12-15].  Both  these  methods  require  object  shape  and  are  based  on 
solving  optimization  problem  which  requires  high  computation,  hence  they  are  hard  to  implement  for  real-time 
applications. 

To  avoid  solving  the  path  planning  problem  for  prosthetic  hands,  many  researchers  advocated  under-actuated 
mechanisms,  which  are  capable  of  adapting  to  object  shape  mechanically  and  without  additional  computation  [16- 
18].  In  these  mechanisms,  the  number  of  actuators  are  less  than  the  DOFs,  and  because  of  less  actuators  they  have 
less  weight.  However  fewer  actuators  result  in  less  functionality,  because  fingers  joints  can‘t  move  independently. 

Arimoto  et  al.  [19]  used  — ktual  spring-damper  hypothesis”  for  control  of  robotic  arm-hand  systems.  A  similar 
method  called  — ktual  model  control”  is  also  suggested  by  J.Pratt  et  al.  [20]  used  for  walking  robots,  and  it  is  based 
on  defining  virtual  forces  between  two  points.  Both  methods  are  based  on  the  use  of  Jacobian  matrix  to  relate  task 
space  movement  to  joint  space.  In  [20]  it  is  shown  that  any  kind  of  force  can  be  defined  between  two  points  and  the 
other  study  [19]  shows  that  use  of  spring-damper  forces  will  result  in  human  like  movement.  From  physiological 
point  of  view,  human  skilled  multi -joint  reaching  movement  has  these  characteristics  that  1)  endpoint  trajectory 
become  a  quasi-straight  line  and  less  variable,  2)velocity  profiles  of  the  endpoint  has  a  bell-shape,  and  3)  joint 
trajectories  are  rather  variable  from  trial  to  trial  [19]. 

In  this  report,  a  new  control  scheme  is  proposed  that  can  efficiently  address  the  problem  of  power  grasping  without 
complete  knowledge  of  the  shape  of  the  object  which  may  be  called  -blind  power  grasping”  for  prosthetic  hand.  The 
proposed  method  is  based  on  the  works  by  Arimoto  et  al.  [19]  using  virtual  spring-damper  (VSD)  hypothesis  used 
for  control  of  robotic  arm-hand  systems.  In  our  report,  we  use  the  above  mentioned  hypothesis,  in  particular  for  the 
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power  grasping  of  a  prosthetic  hand.  In  this  method,  we  define  a  virtual  spring-damper  between  finger  tip  and 
desired  point  for  control  of  movement  of  fingers.  Further,  in  this  method  there  is  no  need  to  introduce  any 
performance  indices  to  solve  inverse  kinematics  uniquely  and  Jacobian  pseudo -inverse  or  inverse  dynamics  which 
are  common  methods  to  define  trajectories  of  redundant  DOFs  robots.  Besides,  in  the  present  method,  there  is  no 
need  for  any  information  on  tactile  or  force  sensing. 

The  report  is  organized  as  follows.  In  Section  2  modeling  of  prosthetic  hand  is  discussed,  Section  3  covers  virtual 
spring  damper  method.  Section  4  describes  control  strategy.  Section  5  analyzes  the  efficiency  of  the  proposed 
control  scheme  using  numerical  simulation.  Finally,  conclusion  and  discussion  are  presented  in  section  6. 


1.2.  Modeling  of  Prosthetic  Hand 

In  this  control  method,  controller  is  not  derived  directly  from  dynamic  model  of  the  system.  Kinematics  equation 
and  Jacobian  matrix  are  the  required  for  controller  design. 

A  model  of  a  robotic  hand  system  is  shown  in  Fig.l.  The  model  consists  of  a  finger  with  3DOF  which  represents 
three  joints  of  a  finger  and  palm. 


Fig.  1 :  Model  of  a  Robotic  Hand  System. 

In  this  report  we  assume  the  following: 

1-  Movement  of  both  finger  and  object  are  confined  to  a  two  dimensional  horizontal  plane,  and  therefore  there  is  no 
gravity  effect. 

2-  The  object  is  assumed  to  be  initially  stable  in  its  position. 

3-  The  initial  movement  toward  object  is  handled  by  amputee,  so  the  hand  is  close  enough  to  the  object  before 
grasping. 

The  position  of  the  tip  of  the  finger  is  evaluated  as  (see  Figure  1): 


x  =  lL  cos  q1  -|-  l2  cos(qt  -|-  .g2)  +-  l2  cos(qt  -|-  q2  4 

(1) 

y  =  h  sin  qi  -h  l2  sin(qi  +-  q2)  +-  I3  sin^q^  +-  q2  -f 

(2) 


where,  llf  l2,  and  l3  are  lengths  of  each  finger  and  qh  q2,  and  q3  are  angles  of  each  corresponding  joint. 
Based  on  above  equation  the  Jacobian  matrix  is  as: 


JlS 

iual 

Uz  L 

hi 

Jz3  T 

/L1  =  sinq L  -  l2  sm(gL  -f  q2 )  -  sin{qt  + 

(4) 

!u  =  ~h  +  Vi)  ~  h  sin(9i  +  <12  +  (5) 
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Iu  =  -h  sin(qL  +  <j2)  -  fa  sin{q,_  +  qz+  <Ja),  (6) 

jvt  =  ~h  smfai  +  g2  +  ,  (7) 

ill  —  ^1  CD£  (?i  +  ^2  COS  0?  L  +  q2)  -h  Eg  CtJS(jJL  -f  q 

(8) 

;22  =  l2  cos(q1  +-  g2)  -|-  I3  cos(gL  -f  g2  +-  gg),  (9) 

723  =  *a  cos(qL-\-q2  -f  fl3>  (10) 


1.3.  Virtual  Spring-Damper  Method 

-Virtual  model  control”  is  a  motion  control  scheme  that  uses  simulations  of  virtual  components  to  generate  desired 
joint  torques  [20].  These  joints  produce  the  same  effect  that  the  virtual  elements  placed  on  robot  would  have  created; 
hence  they  create  the  illusion  that  these  virtual  elements  are  connected  to  the  real  robot.  Virtual  elements  can  be  any 
kind  of  real  physical  elements  such  as  springs,  dampers,  gravity  fields,  nonlinear  fields  or  any  other  components. 
Virtual  model  control  was  proposed  by  J.  Pratt  et  al  [20]  for  biped  walking  robot.  In  a  study  by  Arimoto  [19]  on 
robotic  hand  arm  system,  it  is  shown  that  using  a  virtual  spring  damper  between  robot  end  effector  and  desired  point, 
and  virtual  dampers  at  each  joint,  human  like  movement  can  be  achieved. 

For  power  grasping  by  a  prosthetic  hand,  one  of  the  best  options  is  the  use  of  Virtual  Spring-Damper  (VSD) 
hypothesis.  Some  benefits  of  VSD  control  scheme  are  that  it  has  a  simple  structure  and  requires  relatively  less 
computation.  Besides,  it  doesn‘t  need  inverse  dynamics  to  precisely  define  the  robot  movement.  Thus,  we  use  spring 
set  points  instead  of  commanded  movement  and  robot  automatically  adapts  its  shape.  Since  finger  joints  at 
prosthetic  hand  work  as  virtual  dampers,  which  is  sensitive  to  velocity  and  not  to  position,  they  don‘t  have  a  forced 
shape,  instead  just  finger  tip  follow  a  defined  path  as  will  be  discussed  more  in  control  strategy  section. 

The  joint  torques  to  virtual  forces  is  given  by: 

r=fF,  (11) 

where  x  is  the  torque,  and  F  is  the  force  due  to  virtual  spring  damper  given  as 

F  =  -(fy Hx  +  kAxl  (12) 

and 

^spring -damper  ~  ~1  (■f'V  k.V  F  kji.xj,  (13) 


where  k  represents  the  stiffness  of  the  virtual  spring,  Ax  is  distance  between  finger  tip  and  desired  point,  and  £  is  the 
damping  ratio.  The  damping  force  is  defined  at  each  joint  as 

Tjz-ir.  ts  damping  ~  -Cq,  (14) 

where,  C  denotes  a  diagonal  positive  definite  matrix  as  follows: 

C  =  <0  diag(cL . cn),  (15) 

Hence  control  signal  would  be  sum  of  these  two  terms 

u  -  ~Cq  -  JT (q)Q\fkx  -f  fcAx).  (16) 


Higher  values  of  k  result  in  more  accurate  and  faster  response  to  the  desired  point  and  higher  C  provides  more 
stability.  Thus  k  and  C  are  chosen  as  design  variables. 
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1.4.  Control  Strategy 

Virtual  spring-damper  hypothesis  is  suitable  for  point  to  point  control.  Defining  the  desired  trajectory  as  a  semicircle 
(in  order  to  have  a  hill  grasp  of  the  object)  given  by, 

xd  =  (lL  +  l2  +  Z3)  cos[t}t  (17) 

yd  =  Oi  +  +  Eg]  (18) 


Where  t  is  proportional  to  EMG  signal  which  is  scaled  to  change  between  0<t<7i. 

As  shown  in  Fig.  2,  after  passing  this  semi-circle,  finger  tip  goes  toward  center  to  make  a  tighter  grasping.  This  is 
achieved  by  defining  a  desired  point  close  to  center. 

As  mentioned  earlier,  the  goal  is  not  exactly  following  the  defined  path.  If  the  object  is  big,  due  to  contact  of  hand 
and  object,  it  would  be  impossible  to  follow  exact  path  and  following  this  path  is  just  to  achieve  grasping. 

Fig.  3  illustrates  the  physical  counterparts  of  the  virtual  forces  for  control  strategy  and  Fig.  4  shows  structure  of  the 
proposed  control  system.  As  shown,  the  command  force  comes  from  EMG  signal,  and  controller  provides  the 
movement  for  hand  which  has  dynamic  interaction  with  the  object. 


Fig.  2:  Semi-Circle  Path  of  Finger  Tip 


Desired  Point 


Fig  3:  Physical  Counterparts  of  the  Virtual  Forces 


Fig.  4:  Control  Diagram  of  Prosthetic  Hand  System 


1.5.  Numerical  Simulation 

In  order  to  show  the  effectiveness  of  the  proposed  control  strategy,  numerical  simulations  were  conducted  to  grasp 
three  different  objects,  based  on  the  physical  parameters  of  a  hand  system  and  objects  summarized  in  Table  1. 
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In  order  to  simulate  dynamics  of  the  hand  and  interaction  with  object,  Adams  software  is  used.  This  software  is  a 
multi-body  analysis  simulation  program  that  solves  the  rigid  body  dynamic  equilibrium  equations  and  directly 
interfaces  with  Matlab/Simulink  software  in  order  to  implement  controller.  The  contact  between  object  and  hand  is 
modeled  and  three  sets  of  simulations  with  different  objects  are  performed.  In  all  three  simulations  the  same  control 
strategy  is  used  which  shows  controller  can  handle  grasping  without  information  about  physical  parameters  of 
object.  Fingers  and  objects  are  assumed  to  be  rigid.  In  these  simulations  the  EMG  signal  is  assumed  to  increase 
linearly  with  time. 

Table  1.  Parameters  used  for  simulation 


Index  finger  link  1 

5  cm 

Index  finger  link  2 

2.5  cm 

Index  finger  link  3 

2.5  cm 

Damping  at  joints 

0.01  kg/s 

Virtual  damping  ratio  (c) 

1 

Virtual  spring  stiffness  (k) 

50  N/m 

Rectangular  object  width 

3  cm 

Circular  object  radius 

3  cm 

Star  shape  object  outer  radius 

2  cm 

1.5.1  Simulation  One 

For  first  simulation  a  rectangular  (cubic)  object  is  used,  and  as  mentioned  earlier  the  movement  is  restricted  to  2D 
movement.  Object  is  not  moving  initially.  Hand  starts  movement  from  open  finger  configuration.  The  hand  positions 
at  1  second  time  interval  are  shown.  Fig.  5  shows  finger  tip  angle  with  respect  to  palm. 

As  it  is  shown  in  Fig.  6  the  grasping  is  accomplished  successfully. 


1.5.2  Simulation  Two 

For  the  second  simulation  a  glass  (circular  object)  is  used.  The  control  parameters  are  identical  to  previous 
simulation. 

As  shown  in  Fig.  7  the  grasping  is  done  successfully  and  the  hand  positions  for  1  second  time  interval  are  depicted. 
Similarly  the  finger  tip  angle  respect  to  palm  is  shown  in  Fig.  8. 

Regardless  of  object  shape,  by  use  of  proposed  control  scheme  the  hand  can  successfully  grasp  objects. 
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Fig.  7  (a-h)  :  Grasping  of  a  Glass  (1  sec.  Intervals) 
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1.5.3.  Simulation  Three  ,  . 

For  the  third  simulation  a  star  shape  object  is  used.  The  control  parameters  are  identical  to  ^  '  >  simulation. 

As  shown  in  Fig.  9  the  grasping  is  done  successfully  and  the  hand  positions  for  1  second  time  interval  are  illustrated. 
Final  position  of  grasping  in  3D  (isometric)  view  is  depicted  in  Fig.  10.  Similarly  the  finger  tip  angle  respect  to 
palm  is  shown  in  Fig.  11. 

The  complicated  shape  of  object  shows  that  controller  is  able  to  handle  grasp  for  wide  variety  of  objects,  without 
information  of  object  shape. 


(c) 
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(g) 


Fig.  10  :  3D  (Isometric)  View  of  Star  Shape  Object  After  Grasping 
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Fig.  11:  Fingefs  Tip  Angle  at  Star  Shape  Object  Grasping 


2.  Precision  grasping  control  for  fully  actuated  prosthetic  hand 

Using  the  similar  technique  discussed  in  previous  section  for  power  grasping  which  is  virtual  spring  damper 
hypothesis  a  control  strategy  is  developed  for  precision  grasping.  In  precision  grasping  the  object  is  held  by  tips  of 
the  fingers,  while  in  power  grasping,  the  whole  finger  is  active  and  in  contact  with  the  object.  First  a  model  is 
derived  for  kinematic  of  the  hand,  then  the  control  strategy  and  numerical  simulations  are  provided. 

2.1.  Modeling  of  Prosthetic  Hand 

In  this  control  method,  controller  is  not  designed  based  on  dynamic  model  of  the  system.  Instead,  kinematics 
equation  and  Jacobian  matrix  are  used  for  controller  design. 

A  model  of  a  robotic  hand  system  is  shown  in  Fig.l.  The  model  consists  of  a  finger  with  3DOF  which  represents 
three  joints  of  index  finger,  palm  and  a  finger  with  2DOF  which  represents  thumb. 


Fig.  1.  Schematic  of  a  robotic  hand  system 


In  this  report  we  assume  the  following: 

•  Movement  of  both  finger  and  object  are  confined  to  a  2  dimensional  horizontal  plane,  and  therefore  there  is 
no  gravity  effect. 

•  The  object  is  assumed  to  be  initially  stable  in  its  position. 
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•  The  initial  movement  toward  object  is  handled  by  amputee,  so  the  hand  is  close  enough  to  the  object  before 
grasping. 

The  position  of  the  tip  of  index  fingers  is  evaluated  as  (see  Fig.  1): 

X{  =  liL  cos  qit  -|-  cos(q,:i  -|-  q^)  T-  cos(q,:i  +-  qi2  +-  qi2), 

(1) 

Vi  =  ln  sin  flii  +  fii  sin(<j ii  +  ?ia)  -I-  fia  +  qa  +  <?&),  (2) 

where,  lih  li2,  and  li3  are  lengths  of  index  finger  and  qih  qi2,  and  qi3  are  angles  of  each  corresponding  joint.  Similarly 
the  position  of  thumb  finger  is  evaluated  as: 

xt  =  ln  cos  qn  +  la  cos(qtl  +  qu),  (3) 

Vt  =  fti  sin  qa  +  In  s!n(grl  +  qn},  (4) 


where,  ltl  and  lt2  are  lengths  of  thumb  finger  and  qtl  and  qt2,  are  angles  of  corresponding  joints. 
Based  on  above  equation  the  Jacobian  matrix  for  index  finger  is  as: 


h  = 


It  = 


■  dx 

dx 

dx  1 

0<?ii 

dq 

12 

dy 

dy 

dy 

hga 

dq 

12 

dq^- 

as: 

■  dx 

dx  ■ 

-3-q-rl 

d$tz- 

(5) 


(6) 


2.2.  Control  Strategy 

Virtual  spring-damper  hypothesis  is  suitable  for  point  to  point  control.  In  precision  grasping  two  approaches  can 
be  considered.  1)  Defining  a  virtual  spring  damper  between  fingers  tip  and  geometrical  center  of  the  object,  which 
requires  information  about  the  object  position  and  shape,  and  this  information  is  not  available  in  case  of  a  prosthetic 
hand  for  the  controller  which  is  used  in  [13]  2)  Defining  a  virtual  spring  damper  between  tips  of  two  fingers,  then 
fingers  attract  together  and  grasp  the  object  in  between,  without  exact  knowledge  of  object  position  and  shape.  In 
this  case  the  amputee  should  place  the  hand  close  to  the  object  and  in  appropriate  position.  Besides,  a  virtual  damper 
force  is  considered  at  each  finger  joint.  The  latter  method  is  used  and  physical  counterpart  of  virtual  forces  are 
depicted  at  Fig.  2. 


Virtual  Spring  Damper 


Fig.  2.  Physical  counterparts  of  the  virtual  forces 


Higher  values  of  k  (virtual  spring  stiffness)  results  in  faster  movement  of  fingers  as  well  as,  higher  grasping  force. 
Thus  by  defining  k  proportional  to  EMG  signal,  amputee  have  control  over  speed  of  movement  and  grasping  force. 
The  damping  coefficient  of  finger  joints,  can  change  the  final  shape  of  fingers.  The  joints  with  lower  damping  tends 
to  move  more,  while  higher  damped  joints  move  more.  The  appropriate  values  of  damping  are  evaluated  based  on 
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trial  and  error  to  reach  positions  close  to  normal  hand  and  they  are  held  constant  for  further  simulations. 


2.3.  Numerical  Simulation 

In  order  to  show  the  effectiveness  of  the  proposed  control  strategy,  numerical  simulations  were  conducted  to 
grasp  two  different  objects,  based  on  the  physical  parameters  of  a  hand  system  and  objects  summarized  in  Table  I. 

The  Adams  software  which  is  multi -body  dynamic  simulation  software  is  used  for  numerical  analysis.  The 
software  is  capable  to  conduct  information  between  Matlab/  Simulink  software  environment,  hence  the  plant  is 
modeled  by  Adams  and  controller  is  implemented  in  Matlab/Simulnk. 

TABLE  I 

PARAMETERS  USED  FOR  SIMULATION 


Parameter 

Value 

index  finger  link  1  length 

5  cm 

index  finger  link  2  length 

2.5  cm 

index  finger  link  3  length 

2.5  cm 

thumb  finger  link  1  length 

4  cm 

thumb  finger  link  2  length 

3  cm 

distance  between  thumb  and 

6  cm 

index 

damping  at  joints 

0.01  kg/s 

virtual  damping  ratio 

1 

virtual  spring  stiffness 

50  N/m 

rectangular  object  width 

2  cm 

circular  object  radius 

2  cm 

Fig.  3.  Finger  movements  in  4  different  position  (0.25  sec.  intervals) 

For  first  simulation,  two  fingers  are  modeled  without  any  object  in  between.  As  shown  in  Fig.  3,  two  fingers  come 
together,  and  final  position  is  close  to  normal  hand  coordination. 
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Fig.  4.  Rectangular  object  grasping  (0.25  sec  intervals) 

For  the  second  simulation,  a  rectangular  object  is  chosen  to  be  grasped.  The  object  is  free  to  move  in  2 
dimensional  plane,  and  contact  and  friction  force  are  simulated  between  finger  tip  and  the  object.  The  object  is 
placed  at  arbitrary  final  position  of  previous  experiment.  The  finger  movement  at  0.25  sec  time  intervals  and  finger 
tip  angles  relative  to  palm  are  shown  respectively  in  Figs.  4  and  5. 


Finger  Tip  Angles 


Fig.  5.  Finger  tips  angle  at  rectangular  object  grasping 
(index  finger  solid  line  and  thumb  finer  dashed  line) 


As  it  is  shown  after  contact  with  object  at  approximately  1.5  second  the  angles  are  not  changing  much.  The  small 
changes  are  due  to  object  movements  toward  left. 
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Fig.  6.  Circular  object  grasping  (0.25  sec  intervals) 


For  third  simulation  a  round  object  is  selected.  The  object  is  not  subjected  to  any  constrain  in  2  dimensional 
plane.  The  contact  and  friction  force  are  defined  between  object  and  finger  tip.  The  virtual  spring  coefficient  which 
is  proportional  to  EMG  signal  is  assumed  to  be  constant.  Almost  after  1  sec,  the  fingers  contacted  with  the  object. 
The  finger  movement  at  0.25  sec  time  intervals  and  finger  tip  angles  relative  to  palm  are  shown  respectively  in  Figs. 
6  and  7. 


Finger  Tip  Angles 


Fig.  7.  Finger  tips  angle  at  circular  object  grasping 
(index  finger  solid  line  and  thumb  finer  dashed  line) 


Appropriate  object  position  and  friction  force  between  fingers  and  object  are  important  parameters  that  help 
successful  grasping.  For  the  case  of  a  round  object  if  the  object  has  inappropriate  position  or  friction  is  not  enough, 
the  grasping  might  be  unstable,  but  guarantied  successful  grasping  under  all  conditions  require  information  about 
object  shape  and  position  which  are  not  available  for  case  of  a  prosthetic  hand  hence  this  control  strategy  is 
appropriate  for  most  of  the  daily  activities. 


3.  Underactuated  prosthetic  hand 
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3.1.  Introduction 

In  underactuated  mechanisms  the  number  of  actuators  is  less  than  their  degree  of  freedom.  These  mechanisms  are 
widely  used  in  prosthetic  hands  because  of  two  useful  properties:  the  first  advantage  is  less  weight  due  to  less 
actuator  which  is  used  in  their  design  and  the  second  advantage  is  easier  control  method. 

Underactuation  can  be  implemented  through  the  use  of  passive  elements  like  mechanical  limits  and  springs  leading 
to  a  mechanical  adaptation  of  the  finger  to  the  shape  of  the  object  to  be  grasped.  Underactuated  robotic  hands  are  the 
intermediate  solution  between  fully  actuated  robotic  hands  for  manipulation  and  simple  grippers.  It  takes  advantage 
of  the  mechanical  intelligence  embedded  into  the  design  of  the  hand  allowing  the  shape  adaptation  of  the  fingers.  In 
an  underactuated  finger,  the  actuation  torque  (or  more  generally  wrench)  is  applied  to  the  input  of  the  finger  and  is 
transmitted  to  the  phalanges  through  suitable  mechanical  design,  e.g.  linkages,  pulleys  and  tendons,  gears,  etc. 


Fig  1. Closing  sequence  of  a  two-phalanx  underactuated  finger 
An  example  of  underactuated  two-phalanx  finger  using  linkages  performing  a  typical  closing  sequence  is  illustrated 
in  Fig.  1.  The  finger  is  actuated  through  the  lower  link,  as  shown  by  the  arrow  in  the  figure.  Since  there  are  two 
DOFs  and  one  actuator,  one  (two  minus  one)  elastic  element  is  used.  In  the  first  two  steps  of  the  Figure,  the  finger 
behaves  as  a  single  rigid  body  in  rotation  about  a  fixed  pivot.  When  the  proximal  phalanx  makes  contact  with  the 
object,  the  second  phalanx  is  rotated  away  from  the  mechanical  limit,  and  the  finger  is  closing  on  the  object  since 
the  proximal  phalanx  is  constrained.  During  this  phase,  the  actuator  has  to  produce  the  force  required  to  extend  the 
spring.  Finally,  both  phalanges  are  in  contact  with  the  object  and  the  finger  has  completed  the  shape  adaptation 
phase.  The  actuator  force  is  distributed  among  the  two  phalanges  in  contact  with  the  object.  It  should  be  noted  that 
this  sequence  occurs  with  a  continuous  motion  of  the  actuator.  Notice  also  the  mechanical  limit  that  allows  a  pre- 
loading  of  the  spring  to  prevent  any  undesirable  motion  of  the  second  phalanx  due  to  its  own  weight  and/or  inertial 
effects,  and  to  prevent  hyperextension  of  the  finger.  Springs  are  useful  for  keeping  the  finger  from  incoherent 
motion,  but  when  the  grasp  sequence  is  complete,  they  still  oppose  the  actuator  force.  Thus,  they  should  be  designed 
with  the  smallest  stiffness  possible,  however  sufficient  to  keep  the  finger  from  collapsing. 

The  basic  property  of  the  transmission  system  of  an  underactuated  finger  is  to  offer  n  >  1  DOF  produced  with  fewer 
than  n  actuators.  In  Figure  2,  the  transmission  stage  consists  of  a  five-bar  linkage  (the  base  joint  is  a  double  pivot) 
with  two  DOFs  but  one  angle  is  initially  constrained  to  a  particular  value  with  the  spring  and  the  mechanical  limit. 


3.2.  Force  analysis  of  underactuated  finger 

A  two  phalanx  finger  is  considered  in  Fig.2.  The  input  torque  Tt  is  applied  to  the  link  a  which  transmits  the  torque  to 
the  whole  finger  trough  phalanges.  A  rotational  spring  with  force  T  is  located  at  O  which  moves  the  phalanx  back  to 
its  original  position  in  absence  of  external  force.  The  closing  process  is  shown  in  Fig  2.  Providing  a  mechanical  limit 
can  help  in  order  to  make  pre -tension  for  the  spring  to  prevent  undesirable  motion  of  the  second  phalanx  and  also 
hyper-flexion  of  the  finger. 
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Fig.2.  Underactuated  hand  model  and  exerted  forces 

To  obtain  the  static  model  between  inputs  and  outputs  of  the  finger  the  virtual  work  principle  is  used.  Equating  the 
input  and  output  virtual  powers  results 

tT&»i  =  fv  (1) 

Where  t  is  the  input  torque  vector  exerted  by  the  actuator  and  the  spring,  ttfr  is  the  rotational  velocity  vector,  /  is  the 
vector  of  contact  forces,  and  v  is  the  velocity  of  the  contact  points  along  the  normal  vector  of  each  phalanx.  Contact 
forces  are  assumed  to  be  normal  to  the  phalanges  and  without  friction.  Each  element  in  the  above  equation  can  be 
expressed  as 

t  =  \  *  1 


F  = 


■ia 

_  Riyi] 

'  b&y2  J 

The  normal  velocities  of  the  contact  point  can  be  expressed  as  a  Jacobian  matrix  }7  and  the  derivatives  of  the 
phalanx  joint  coordinates  which  is  a  natural  choice,  as  v  =  JT  §  or 


(2) 

(3) 

(4) 

(5) 


yl  \  _ 
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\  h 
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Through  differential  calculus,  one  also  can  relate  vector  to  the  derivatives  of  the  phalanges  joint  coordinates 
defined  previously  with  an  actuation  Jacobian  matrix  JA  as  5  =  }&&*&  or 

W 


Where  X=1  and 


Finally,  one  obtains 


a 


Y  = 


=  \x  Hf*1 

[0  lJ  [&% 


€  [[j  an;gz  -ftp  — i  sin  f 
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f  =  J¥TJlTt 


(7) 


(8) 


(9) 


which  is  the  equation  that  provides  a  practical  relationship  between  the  actuator  torque  and  contact  forces.  If  the 
spring  contribution  is  neglected  the  analytical  expression  are  rather  simple  linear  functions  of  the  actuator  torque, 


c  it*  k2  (cot  ccs  e1+  sin 

f  — - - - y 

12  €.  k2  (cot  cos  ETi  +  sin  -Ei  )  c 


(ii) 
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where  h  =  e( cos(&2  — VO  —  sinf^  —  ip)  cot  £?')  is  the  distance  between  point  0Land  the  intersection  of  lines 

(OOi)  and  P2 ).  Also,  c^is  the  angle  between  link  a  and  the  first  phalanx.  It  can  be  shown  that 

,  ^  „  sinx  ^ 

cot  8  cos  a  h  F  sin  a  *  —  - — -  (12) 

^  sm  S  y  7 

where  X  is  the  angle  between  links  a  and  b. 


3.3.  Design  optimization 


After  obtaining  the  forces  equation  two  consideration  form  the  guidelines  for  design  parameters  selection:  first  grasp 
should  be  stable  which  means  ejection  should  be  prevented  and  differences  between  the  phalanx  forces  should  be 
minimum  possible  value.  In  Ejection  phenomena  the  finger  slide  and  push  the  object  out  instead  of  a  secure 
grasping.  To  prevent  ejection  the  exerted  forces  by  each  phalanx  to  the  object  should  be  positive.  It  is  also  desirable 
that  forces  at  each  phalanx  to  be  close  to  their  mean  value  and  force  distribute  evenly  between  phalanxes  which  is 
referred  to  as  force  isotropy.  So  the  conditions  which  should  be  satisfied  can  be  expressed  as: 


(13) 


f  A  =  A 

l/i  >  0  and  f2  >  0 

In  order  to  have  force  isotropy  it  is  necessary  that  forces  by  each  phalanx  are  equal,  but  forces  are  function  of 
contact  position  and  &2  angle  hence  it  is  a  local  property.  Therefore  if  the  object  moves  these  conditions  are  not  true 
anymore.  However  in  this  case,  another  step  is  necessary  since  h  is  a  function  of  the  design  parameters  and  the  angle 
02 .  Furthermore,  many  design  variables  are  available  to  satisfy  the  latter  equation,  namely  a,  b,  c,  and  d. 

For  instance,  if  one  chooses  a  =  b,  a  known  c  (e.g.  resulting  from  minimal  distance  considerations)  and  ip  =  7,  a  is 
completely  defined  as 

_  2  i±€  sin  92+£2  +  [i  VA 


with  , 


A  =  (C  -  1)((C  -  l)c2  F  2cClLsin&2)  F  C2lf 


(14) 


(15) 


B  =  —  c2  F  CcL  —  ltesm&2  F  2Cltcsm&2  F 

_ 


c  =  —K&7  f 


kz  +  i±  era  9Z  + 


(16) 

(17) 


The  above  mentioned  relationship  can  be  used  in  order  achieving  isotropic  design.  However,  the  isotropic  property 
is  not  very  robust  with  respect  to  design  parameters,  so  it  suggests  using  the  following  method. 

If  one  obtains  an  isotropic  and  therefore  stable  design  for  a  particular  contact  set  (kvk2,  &2  ),  it  may  be  of  interest 
that  the  finger  is  also  robust  with  respect  to  ejection  around  this  isotropic  point,  in  order  to  ensure  that  a  deviation 
from  this  configuration  does  not  lead  to  an  unstable  grasp.  The  final  aim  is  to  guaranty  stability  for  all  grasps  if 
possible  and  satisfy  certain  -quality”  based  indices  like  the  isotropy.  An  index  that  can  be  used  to  ensure  the  grasp 
stability,  even  if  the  proximal  contact  is  lost,  is: 

jU.  =  — - — - ' — 1  (18) 

f^dk^dG^ 

Where  S{k2  ,-32)  is  a  Kronecker-like  symbol  for  characterizing  the  stability  of  the  contact  situation: 
fGc.,ftO=[ lif  the  final  grasp  is stabb 
“  “  L  0  otherwise 

This  index  is  the  ratio  between  the  stable  and  unstable  areas  in  the  grasp -state  plane  of  the  finger.  Contour  plots  of 
the  index  fi  is  illustrated  in  Fig  3.  for  a  mechanically  actuated  finger.  The  optimal  design  parameter  values  can  be 
obtained  using  the  following  plot.  In  our  design  equal  lengths  are  considered  for  both  phalanxes  (l2/li=l),  so  the 
optimal  value  for  c/a  is  around  0.6. 
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Performance  index 


Fig  3.  Performance  index  for  linkage  driven  2  phalanx  underactuated  hand3.4.  Control  system 

The  control  is  aimed  at  exploiting  the  main  properties  of  underactuation  to  perform  motion  tasks  close  to  reference 
angles  obtained  by  EMG  signal.  The  ordinary  task  considered  for  design  and  development  of  the  motion  control  law 
is  the  finger  preshaping  for  the  palmar  grasp  of  a  cylindrical  object. 

Kinematic  coupling  among  the  joints  is  related  by  the  relation 

*,  =  IA  +  W*1  +  %)  (20) 

*3  =  +■  A)  (21) 

The  dynamic  relation  among  the  joints  are  derived  in  equations  (10)  and  (11). 

3.4.1.  PD  Control  in  the  joint  space  with  elastic  compensation 

Dynamic  relation  (20)  and  (21)  is  used  to  actively  control  the  first  joint  and  passively  move  the  second  joint.  The 
proposed  control  law  is  a  modified  version  of  the  standard  PD  control  in  the  joint  space  with  gravity  compensation 
and  is  expressed  as 

Ti=KpS-KDd+g(0)+TE  (22) 

Where  &  =  &D  —  0  is  the  joint  position  error  defined  as  the  difference  between  the  reference  set  point  G^and  the 
current  joint  angle  &,$(.&)  is  the  estimation  of  joint  gravitational  torque,  and  and  KD are  the  diagonal  gain 
matrices  for  the  proportional  and  derivative  control  actions,  respectively.  In  addition  to  the  standard  PD  control  plus 
gravity  compensation,  an  elastic  term  is  introduced  in  order  to  compensate  for  the  preload  spring  located  between 
phalanxes  joint. 

The  joint  elastic  torque  is  expressed  as  Ts  =  kA82 . 

3.5.  Numerical  simulation 

Based  on  results  of  optimal  design  a  model  of  two  phalanx  finger  is  made  in  Adams  software  linked  with 
Matlab/Simulink,  which  is  shown  in  Fig.  3.  Both  phalanxes  has  the  equal  length  and  c/a  ratio  is  0.6  based  on  optimal 
design  analysis.  Torsion  spring  is  located  between  two  phalanxes.  The  revolute  joint  is  located  at  joints  and  a  motor 
is  moving  the  finger  which  is  shown  by  round  arrow.  A  circular  object  is  chosen  to  be  grasped  and  the  proposed 
controller  is  used  to  control  finger  movement. 
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Fig.3.  The  two  phalanx  finger  modeled  in  Adams  software 


By  applying  the  control  algorithm  to  the  mechanism  the  finger  can  grasp  the  object.  The  grasping  sequence  is  shown 
in  Fig.  4  . 


Fig.  4.  The  grasping  sequence  of  a  circular  object  by  underactuated  hand 


The  finger  tip  angle  respect  to  horizontal  plane  is  shown  in  Fig.  5.  It  starts  from  90  degree  and  after  first  phalanx  has 
contact  with  the  object  the  rate  of  change  of  angle  is  changed. 
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90.0 


fingertip  angle  respect  to  horizontal  plane 


Fig.  5.  Finger  tip  angle  respect  to  horizontal  plane  for  underactuated  hand 

Angle  between  two  phalanxes  is  shown  in  Fig.  6.  The  angle  starts  from  180  degree  and  gradually  changes  until 
second  phalanx  touches  the  object. 


Angle  between  two  phalanxes 
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Fig.6.  Angle  between  two  phalanxes  for  underactuated  hand 


3.6.  Comparison  study  between  fully  actuated  and  underactuated  hand 

Two  general  design  which  are  fully  actuated  and  underactuated  and  appropriate  control  methods  for  each  of  them 
are  provided  for  prosthetic  hand.  In  order  to  compare  the  functionality  of  these  two  methods  a  fully  actuated  finger 
with  the  same  dimension  and  with  a  similar  object  to  grasp  is  simulated.  Power  grasping  algorithm  which  described 
before  is  used  in  order  to  control  the  movement.  The  sequence  of  movement  is  shown  in  Fig.  7.  The  finger  tip 
angle  respect  to  horizontal  plane  is  shown  in  Fig.  8  and  the  angle  between  two  joints  is  shown  in  Fig.  9. 
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Fig.  7.  The  grasping  sequence  of  a  circular  object  by  a  fully  actuated  hand 


90.0 

85.0 

80.0 

75.0 

70.0 

65.0 

60.0 

55.0 

50.0 

45.0 

0.0  0.5  1.0  1.5 

Analysis:  Last_Run  Time  [sec}  2011-05-16  05:21:06 

Fig.  8.  Finger  tip  angle  respect  to  horizontal  plane  for  fully  actuated  hand 
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Fig.9.  Angle  between  two  phalanxes  for  hilly  actuated  hand 
As  we  can  see  both  methods  are  capable  to  perform  grasping  and  the  graphs  are  very  similar  for  both  methods. 
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Goal  2B:  Embedded  Hierarchical  Real-Time  Systems 

Overview:  Narrative  that  briefly  (one  paragraph)  describes  the  subject ,  purpose  and  scope  of  the  research .  The 
annual  work  under  this  Goal  involved  the  following  key  tasks:  (1)  the  construction  of  a  test  bed  that  integrates  the 
software  tool  chain  and  prototype  hardware  (test  bed);  (2)  the  study  on  design  and  Implementation  of  the  real-time 
control  strategies  on  the  embedded  test  bed,  and  the  study  and  experimental  set  up  of  a  real-time  operating  system 
on  the  embedded  processor  used  in  (1);  (3)  the  study  of  integrating  sEMG  models  on  the  embedded  test  bed  for  real¬ 
time  control,  signal  processing  and  acquisition.  The  primary  purpose  of  this  work  is  to  develop  an  embedded  control 
system  by  researching,  designing,  testing,  and  prototyping  the  hardware  and  software  for  the  system. 

This  section  of  the  report  shall  describe  the  research  accomplishments  associated  with  each  task  outlined  in  the 
approved  Statement  of  Work.  Data  presentation  shall  be  comprehensive  in  providing  a  complete  record  of  the 
research  findings  for  the  period  of  the  report.  Provide  data  explaining  the  relationship  of  the  most  recent  findings 
with  that  of  previously  reported  findings.  Appended  publications  and/or  presentations  may  be  substituted  for 
detailed  descriptions  of  methodology  but  must  be  referenced  in  the  body  of  the  report.  If  applicable,  for  each  task 
outlined  in  the  Statement  of  Work,  reference  appended  publications  and/or  presentations  for  details  of  result  findings 
and  tables  and/or  figures.  The  report  shall  include  negative  as  well  as  positive  findings.  Include  problems  in 
accomplishing  any  of  the  tasks.  Statistical  tests  of  significance  shall  be  applied  to  all  data  whenever  possible. 
Figures  and  graphs  referenced  in  the  text  may  be  embedded  in  the  text  or  appended.  Figures  and  graphs  can  also  be 
referenced  in  the  text  and  appended  to  a  publication.  Recommended  changes  or  future  work  to  better  address  the 
research  topic  may  also  be  included,  although  changes  to  the  original  Statement  of  Work  must  be  approved  by  the 
Army  Contracting  Officer  Representative.  This  approval  must  be  obtained  prior  to  initiating  any  change  to  the 
original  Statement  of  Work. 

Initial  System  Design 

The  design  is  a  multi-level  hierarchical  model  with  a  rectified  sine  wave  input.  A  flash  memory  interfaces  with  the 
high-level  configuration,  making  it  capable  of  holding  the  input  functions  and  the  control  strategy.  The  flash 
memory  interface  is  done  by  a  Secure  Digital  (SD)  card  which  enables  quick  retrieval  and  processing  of  data.  The 
32-bit  PIC,  memory  interface  and  the  Fuzzy  controller  constitutes  the  higher-level  configuration.  This  level  controls 
the  lower-level  configuration,  depending  on  the  input  functions.  Two  16-bit  PICs,  each  associated  with  a  stepper 
motor,  makes  up  the  lower-level  configuration.  Two  lower-level  PICs  are  programmed  to  control  the  stepper  motors. 
The  communication  system  in  this  design  is  unique.  The  two  levels  communicate  through  a  full  duplex  distributed 
communication  system.  This  communication  system  enables  an  effective  coordination  between  the  levels  by 
distributing  the  traffic  and  therefore  eliminates  delay  caused  by  any  traffic  congestion.  This  design  provides 
flexibility  in  handling  with  a  wide  range  of  communication  systems.  This  includes  both  the  hard-wired  and  wireless 
communication  based  on  the  application.  The  stepper  motors  provide  the  necessary  step  changes  in  the  output 
displacement  based  on  the  control  signal  which  is  a  function  of  the  input.  Figure  1  depicts  the  overall  design. 
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Figure  1 :  Block  diagram  of  the  design 

Based  on  the  input  function,  the  fuzzy  controller  of  the  high-level  configuration  has  its  control  signal,  to  which  the 
32-bit  PIC  has  access  to  it  through  flash  memory.  The  32-bit  PIC  sends  the  encrypted  data  via  the  hill  duplex 
distributed  communication  channel  to  access  the  lower-level  PICs.  The  encrypted  data  access  the  two  18-bit  PICs  to 
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control  the  two  stepper  motors  for  the  desired  angular  displacements.  The  encrypted  data  has  all  the  information  for 
the  control  signal.  It  also  contains  the  information  regarding  the  desired  step  changes  and  the  directions  of  them.  The 
lower-level  PICs  are  preprogrammed  to  control  the  stepper  motors  with  necessary  angular  displacement. 
Synchronization  plays  an  important  role  in  the  lower-  level  configuration.  The  encrypted  data  and  the 
communication  system  make  the  synchronization  easier.  This  type  of  communication  system  avoids  traffic  delay 
time.  The  encrypted  data  also  has  the  information  about  the  direction  angular  displacement  for  each  motor.  The  key 
part  of  the  model  is  the  fuzzy  controller.  This  gives  the  control  signal  to  the  high-level  controller,  which  encrypts  the 
data  and  send  to  the  lower-level  configuration  the  actuation  signals  for  the  motors. 

The  lower-level  configuration  involves  two  16-bit  PICs  that  contain  the  code  to  control  respective  stepper  motors. 
When  a  message  is  received  from  the  higher  level,  it  will  state  which  motor  it  wants  to  activate  and  what  angular 
displacement  it  should  make.  One  or  both  of  the  PIC  16  will  take  its  message  and  extract  the  angular  displacement. 
Once  this  is  done,  the  PIC  will  relay  a  message  to  the  stepper  motor,  making  it  turn  to  the  desired  position.  The 
PIC  16  will  then  send  a  message  to  the  PIC32  that  lets  the  PIC32  know  that  it  has  completed  its  task.  This 
information  will  be  fed  into  the  fuzzy  controller. 

The  high-level  configuration  uses  a  PIC32.  The  PIC32  has  an  SD  card  reader  attached  to  it.  Having  an  SD  card  will 
allow  large  files  to  be  read  into  the  PIC.  Another  key  part  of  the  high-level  configuration  is  a  fuzzy  controller.  The 
fuzzy  controller  will  be  used  to  analyze  the  data  and  process  it.  The  rectified  sine  wave  is  compared  to  the  set  of 
rules  from  the  membership  function  of  the  intelligent  controller.  Once  a  result  is  determined,  the  controller  will 
send  a  message  to  the  lower-level  controller  [1]. 

[1]  C.  Potluri,  P.  Kumar,  J.  Molitor,  M.  Anugolu,  A.  Jensen,  K.  Hart,  and  S.  Chiu,  — Mlti-Level  Motor  Control  for 
Prosthesis,”  in  the  proceedings  of  the  2010  International  Conference  on  Embedded  Systems  and  Applications  (ESA, 
07/2010). 

Intiai  Controller  Design  (Related  to  EMG) 

In  this  design,  entropy  is  taken  as  the  threshold  value  to  initiate  the  motion  by  the  fuzzy  controller.  The  entropy 
values  are  extracted  from  the  EMG  signals  of  the  index,  middle  and  ring  fingers,  respectively  when  a  power  grip  is 
done.  Table  1  shows  the  entropy  values  for  the  EMG  data  taken  for  11  different  subjects.  Ring!  and  Middlei  in  the 
Table  1  correspond  to  the  entropy  of  the  respective  finger  right  next  to  its  motor  point  location.  Table  2  gives  the 
threshold  values.  The  threshold  values  are  calculated  as  the  difference  between  the  mean  and  twice  the  standard 
deviation.  The  reason  for  taking  the  difference  as  the  threshold  value  is  that  the  controller  should  be  initiated,  even  if 
the  sensors  are  misplaced  from  the  motor  units  or  to  eliminate  the  problems  that  are  caused  by  the  difficulty  in 
identification  of  motor  units  in  some  cases. 


Table  1:  Entropy  values  of  EMG  signals  for  different  fingers 


Index 

Ring 

Middle 

Ring; 

Middlei 

2.9792 

2.9271 

2.1554 

2.3994 

1.6711 

2.6783 

3.0008 

1.9376 

2.3947 

1.5346 

3.4943 

2.9556 

3.0277 

2.9568 

2.6673 

3.679 

3.1714 

2.9268 

3.1175 

3.0581 

3.2110 

2.7015 

1.9228 

2.7997 

1.2993 

2.8451 

2.9429 

1.9555 

3.0541 

1.1095 

1.4653 

1.5559 

3.7151 

2.4827 

1.6919 

1.9371 

1.4639 

3.6648 

3.2785 

1.7955 

2.7399 

3.7132 

3.0820 

3.0425 

2.3929 

2.5421 

3.5961 

2.8593 

2.6782 

2.2040 

Table  2:  Threshold  values  for  index,  ring  and  middle  finger  EMG  signals 


Index 


Ring 


Middle 
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1.3724 


1.5176 


1.522 


In  this  design  [2],  entropy  values  are  used  to  establish  a  threshold  value  for  initiation  of  finger  motions.  The  fuzzy 
logic  determines  the  correlation  between  the  entropy  and  the  corresponding  force.  The  Mamdani  type  fuzzy  logic  is 
used  for  this  design.  The  fuzzy  logic  implemented  has  four  inputs  and  two  outputs.  The  inputs  are  the  threshold 
values  for  individual  fingers  (index,  ring  and  middle)  and  the  entropy  values.  The  output  of  the  fuzzy  logic  are  the 
control  signals  to  initiate  the  motion  based  on  the  threshold  values  and  the  control  signal  for  the  force  based  on  the 
entropy  values.  Figure  2  shows  the  entire  rule  base  for  the  developed  fuzzy  logic,  which  are  given  below: 

1 .  If  4  is  /t,  M[  is  Aft ,  R i-  is  Rr  then  the  control  signal  is  power  grip  close. 

2.  If  I  [  is  not  It,  Mi  is  not  Mt  and  R{  is  not  tfr,  then  control  signal  is  power  grip  open. 

3.  If  4  is  /r  then  control  signal  is  index  finger  close. 

4.  If  4  is  not  4  then  control  signal  is  index  finger  open. 

5.  If  Mj;  is  Aff  then  control  signal  is  middle  finger  close. 

6.  IfAf,;  is  not  Mt  then  control  signal  is  middle  finger  open 

7.  If  Ri  is  R{  then  control  signal  is  ring  finger  close. 

8.  If R[  is  not  then  control  signal  is  ring  finger  open. 

9.  If  entropy  is  low,  then  control  signal  is  low  force. 

10.  If  entropy  is  medium,  then  the  control  signal  is  medium  force. 

11.  If  entropy  is  high,  then  the  control  signal  is  high  force. 

where  4,  ^4,4'  are  the  input  entropy  values  and  4?  are  the  threshold  values  for  individual  index,  middle 

and  ring  fingers.  In  Figure  2,  columns  1,  2,  3  show  the  rules  for  the  sigmoidal  membership  function  of  input,  i.e. 
index,  middle  and  ring  fingers  respectively,  and  the  corresponding  threshold  values.  The  sigmoidal  function  enables 
the  initiation  of  movement  for  an  entropy  value  which  is  greater  than  or  equal  to  threshold  value  of  individual 
fingers.  The  4th  and  6th  columns  give  the  rule  base  for  the  correlation  of  entropy  with  the  corresponding  force.  The 
force  is  classified  by  the  entropy  range  into  three  levels. 
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Figure  2:  Rules  for  the  Mamdani  type  fuzzy  control 


[2]  C.  Potluri,  P.  Kumar,  M.  Anugolu,  S.  Chiu,  M.  Schoen,  A.  Urfer,  and  D.  S.  Naidu,  — AiANFIS-Based  Fuzzy 
Controller  for  Prosthetic  Hand,”  in  the  proceedings  of  the  3rd  IEEE  RAS-EMBS  International  Conference  on 
Biomedical  Robotics  and  Biomechatronics  (BioRob,  09/2010). 


Test  Bed  for  Embedded  Systems  Prototyping 
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A  test  bed  based  on  a  32-bit  embedded  processor  (a.k.a.  programmable  intelligent  controller,  or  PIC)  was  built. 
Figure  3  depicts  the  test  bed,  where  Stage  1  represents  the  (micro -level)  signal  processing  circuitry  composed  of  the 
processor,  its  I/O  expansion  board,  and  the  wiring  needed  to  receive  input  sEMG  signals  and  produce  output 
actuation  signals  for  the  plant  (the  controlled  one-finger  device).  Stage  2  consists  of  circuit  elements  and  wiring  for 
motor  powering,  actuation,  and  isolation  of  the  motor  circuit  from  the  Stage  1  circuit  for  protection  of  the  embedded 
processor  against  power  surge. 


Figure  3.  Overview  of  the  test  bed  (omitting  the  software  tool  chain  computer) 


For  Stage  1,  a  Microchip  PIC32-I  was  used  to  implement  our  system.  Based  on  this  processor,  we  selected  the  dsPIC 
block  set  to  generate  the  C  code  for  PIC32-I  from  Simulink.  The  intent  of  this  approach  for  the  tool  chain  is  due  to 
the  fact  that  our  controller  designs  are  being  specified  in  Matlab  models;  and  Simulink,  along  with  its  Real-Time 
Workshop  toolbox,  provides  a  logical  and  efficient  path  from  model  to  prototype  implementation  and  testing,  for  a 
given  controller  design.  The  processing  flow  of  the  various  elements  of  the  tool  chain  is  shown  in  Figure  4,  which 
builds  and  releases  the  executable  of  the  controller  to  the  PIC. 
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Figure  4.  Processing  flow  for  generating  code  to  the  processor 

Several  hardware  modules  within  the  PIC32-I  were  also  used,  including  the  Analog  Input  module,  the  Digital 
Output  module,  the  Output  Compare  module,  and  the  UART  module.  The  Analog  Input  module  was  used  for 
reading  the  input  sEMG  as  well  as  sensory  feedback  data  from  the  prototype  hand.  PIC32-I  has  an  internal  analog  to 
digital  converter.  A  real-time  control  strategy  uses  the  data  from  the  Analog  Input  module  as  a  reference  signal  or 
the  set  point.  Based  on  the  control  strategy,  the  Digital  Output  module  generates  digital  control  signals.  The  Output 
Compare  module  then  generates  a  pulse  width  modulated  (PWM)  waveform  with  a  specific  duty  cycle,  based  on  the 
input  analog  signal  (i.e.  the  sEMG).  The  UART  module  is  used  to  transmit  the  signals  from  PIC32-I  to  a  personal 
computer  via  the  serial  (i.e.  RS-232)  communication.  This  interface  enabled  us  to  debug  and  evaluate  the 
performance  of  control  strategies. 

For  Stage  2,  we  used  the  SN754410  quadruple  half-H  driver  to  actuate  the  motor  with  the  control  signal.  The  pin 
configuration  for  the  H-driver  is  shown  in  Figure  5.  The  PWM  signal  from  the  Output  Compare  module  is  connected 
to  pinl  (labeled  — 1-,2EN”)  of  the  H-driver.  The  digital  outputs  of  the  PIC32-I  are  connected  to  the  direction  pins  of 
the  H-driver  (pins  2  and  7).  By  switching  the  digital  outputs  to  0  and  1  between  the  direction  pins,  the  motor  will 
rotate  clockwise  or  counter-clock  wise.  Vcci  and  Vcc2  are  connected  to  the  5-volt  DC  supply  of  the  PIC32-I  I/O 
expansion  board. 
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Figure  5.  Pin  configuration  of  the  quadruple  half-H  driver. 

The  test  bed  and  its  operation  was  demonstrated,  in  the  form  of  recorded  video,  to  the  researchers  at  the  Applied 
Physics  Laboratory  of  Johns  Hopkins  University,  a  major  collaborator  on  the  TATRC  project  with  which  ISU  is  a 
part  of.  The  JHU  APL  researchers  expressed  their  interest  in  further  collaboration  with  ISU  in  several  areas 
including  this  and  the  validation/verification  of  sEMG  signal  processing  algorithms. 

Design  and  Implementation  of  Real-Time  Control  Strategies: 

In  this  work  [3],  the  threshold  values  establish  a  correlation  between  the  sEMG  and  the  corresponding  force  required 
for  the  power  grasp  of  an  object.  C.Potluri  et.  al  [2]  explained  the  correlation  between  sEMG  entropy  threshold 
values  and  skeletal  muscle  forces.  These  threshold  values  are  fed  as  the  reference  or  a  set  point  to  the  controller.  The 
signal  flow  is  depicted  as  a  block  diagram  in  Figure  6. 
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Figure  6.  Block  diagram  of  the  real-time  control  signal  flow. 

The  force  feedback  signal  is  acquired  by  a  force  sensitive  resistor  (FSR).  The  FSR  is  mounted  on  the  fingertip  of  the 
prosthetic  hand  prototype  as  shown  in  Figure  8.  The  error  is  computed  as  the  difference  between  the  reference 
value  and  the  actual  force  from  the  FSR.  A  proportion-integral  (PI)  controller  is  employed  to  generate  the  actual 
force  from  the  FSR,  and  make  it  equal  to  the  reference  force  in  order  to  reduce  the  error  & (T)  to  zero.  The  PI  control 
action  is  given  below  in  Equation  (1). 

uit)  -  K0  *  e(t)  -F  e(f)dr,  (1) 

In  Equation  (1),  u(t)  is  the  control  signal,  AT3  is  the  proportional  gain,  and  K{  is  the  integral  gain,  ff(Ois  the  error, 
and  t  is  the  dummy  integration  variable.  Gain  parameters  and  K{  are  tuned  using  the  Ziegler-Nichols  method  in 
MATLAB®. 

Our  proposed  design  was  implemented  on  a  PIC32-I  microcontroller  in  two  stages:  -Signal  Processing”  and  -Alotor 
Actuation.”  The  Signal  Processing  stage  facilitates  the  implementation  of  real-time  control  strategies.  A  dsPIC 
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blockset  was  used  to  generate  the  C  code  for  the  PIC32  from  Simulink®.  The  dsPIC  blockset  generates  a  .hex  file, 
and  this  file  is  imported  by  MPLAB®  to  program  the  PIC. 

Signal  Processing  Stage 

We  used  the  following  modules  of  the  PIC32  as  a  part  of  implementing  our  control  strategies:  the  Analog  Input 
module,  the  Digital  Output  module,  the  Output  Compare  module,  and  the  UART  module.  The  Analog  Input  module 
was  used  for  reading  the  sEMG  and  sensory  feedback  data.  The  PIC32  has  an  internal  analog  to  digital  converter 
(ADC)  which  has  a  1 0-bit  resolution  so  that  it  can  distinguish  up  to  1 024  different  voltages,  usually  in  the  range  of  0 
to  3.3  volts,  and  it  yields  3mV  resolution.  A  real-time  control  strategy  was  developed  using  the  threshold  value  of 
the  entropy  of  sEMG  data  as  a  reference  or  set  point.  The  difference  between  the  threshold  of  the  sEMG  entropy  and 
the  FSR  feedback  was  fed  in  to  the  PI  control  loop  to  reduce  the  error  to  zero.  The  Digital  Output  module  of  the 
PIC32  was  used  to  generate  digital  control  signals.  Depending  on  the  error,  a  pulse  width  modulated  (PWM)  wave 
with  a  specific  duty  cycle  was  generated  in  the  Output  Compare  module.  The  UART  module  in  the  PIC32  was  used 
to  transmit  the  signals  from  the  microcontroller  to  the  PC  via  serial  communication.  In  this  design,  a  virtual  com  port 
was  created  to  feed  the  data  via  USB  cable  to  the  computer.  MATLAB®  was  used  to  read  the  signals  from  the  ports. 
This  enables  the  user  to  troubleshoot  and  see  the  performance  of  the  designed  control  strategy.  The  design  is 
consistent  with  our  initial  system  design  and  the  embedded  test  bed. 

Motor  Actuation  Stage 

A  SN754410  quadruple  half-H  driver  is  used  to  actuate  the  motor  with  the  corresponding  control  signal.  Fig.  3 
shows  the  schematic  of  the  pin  configuration  of  the  half-H  driver.  The  PWM  signal  from  the  Output  Compare 
module  was  connected  to  pinl  (1,  2EN)  of  the  H  driver.  The  PWM  wave  enables  this  H  driver.  The  speed  of  the 
motor  depends  on  the  duty  cycle  of  the  PWM  wave  from  the  Output  Compare  module,  which  is  a  function  of  &(£). 
The  digital  outputs  of  the  PIC32  microcontroller  are  connected  to  the  direction  pins  of  the  Hdriver  (pins  2  and  7). 
Switching  the  digital  outputs  to  0  and  1  between  the  pins  will  makes  the  motor  rotate  in  clockwise  and  counter¬ 
clockwise  directions.  Vcci  and  Vcc2  are  connected  to  the  5V  supply  of  the  PIC32  I/O  board.  This  proposed  design  was 
tested  on  an  index  finger  of  a  prosthetic  hand  prototype.  This  part  of  the  work  made  use  of  the  embedded  test  bed  as 
previously  described. 

Mechanical  Structure  of  Hand  Prototype 

The  robotic  hand  prototype  finger  has  three  degrees  of  freedom,  and  is  actuated  by  two  Pololu  35:1  mini  metal  gear 
motors  and  a  bevel  gear  transmission  system.  The  main  characteristic  of  this  prototype  is  its  biologically-inspired 
parallel  actuation  system  based  on  the  behavior/strength  space  of  the  ’’Flexor  Digitorum  Profundus  (FDP)”  and  the 
’’Flexor  Digitorum  Superficialis  (FDS)”  muscles  as  explained  in  Figure  7.  However  this  research  work  focuses  on 
the  -Region  1”  actuation  system  and  it  is  interfacing  with  a  microcontroller.  Region  1  is  populated  by  the  more 
frequent  dexterous  tasks. 


Figure  7.  Strength  space  of  FDS  and  FDP  muscles 

In  our  design,  the  movements  associated  with  Region  1  are  achieved  by  2  DC  motors.  The  motor  in  the  metacarpal 
phalange  of  the  finger  actuates  the  PIP  (Proximal  Inter  Phalangeal)  joint  and  the  motor  at  the  base  of  the  finger 
actuates  the  MCP  (Meta  Carpo  Phalangeal)  joint,  as  shown  in  Figure  8.  As  the  main  objective  of  this  work  is  to 
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interface  and  control  the  two  motors  with  a  microcontroller,  we  have  not  considered  the  belt  transmission  system 
which  is  designed  to  drive  the  DIP  (Distal  Inter  Phalangeal)  joint  as  well. 


Figure  8.  Region  1  actuation  scheme  for  the  finger 


[3]  C.  Potluri,  Y.  Yihun,  P.  Kumar,  J.  Molitor,  S.  Chiu,  D.S.  Naidu,  S.  H.  Mousavinezhad,  -sEMG  Based  Real- 
Time  Embedded  Force  Control  Strategy  for  a  Prosthetic  Hand  Prototype,”  Accepted  in  2011  IEEE  International 
Conference  on  Electro/Information  technology  (IEEE  EIT,  May  2011). 

Reportable  Outcomes 

Results  are  acquired  from  the  microcontroller  through  UART  channel  2  of  the  PIC32  by  a  virtual  com  port  via  USB 
at  57600  baud  rate.  The  data  from  the  microcontroller  is  converted  into  unit  16  data  before  it  is  transmitted  through 
the  UART.  The  PIC32  microcontroller  is  running  80  million  instructions  per  second  (MIPS)  with  its  phase  lock  loop 
(PLL)  activated.  It  is  running  at  an  external  clock  frequency  of  8MHz  with  internal  scaling  enabled. 


3  3.5  4  4.5  5  5.5  6  6.5  7  7.5 

Time(ms) 


Figure  9.  Feedback  force  and  error  before  grasp 
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Figure  9  depicts  the  feedback  force  from  the  FSR  and  the  error  p(0,  before  the  grasp.  As  there  is  no  contact 
between  the  object  and  the  hand,  the  force  is  low  and  ideally  should  be  zero.  However,  because  the  PIC32‘s  ADC 
module  has  a  high  resolution,  it  shows  some  values  even  when  the  object  is  not  in  contact. 


Time(ms)  xio4 


Figure  10.  Feedback  force  and  error  after  grasp 

Figure  10  depicts  the  feedback  force  from  the  FSR  plotted  against  the  error  &(£)  after  the  grasp.  The  error  eft),  i.e. 
the  difference  between  the  reference  force  from  the  sEMG  threshold  value  and  the  actual  force  feedback  from  the 
FSR,  is  almost  zero.  From  Figure  10,  it  is  also  evident  that  the  actual  force  from  the  FSR  is  almost  equal  to  the 
reference  force  value  that  is  established  from  the  sEMG  threshold  value,  as  the  error  approaches  zero.  Figure  1 1 
shows  the  computational  load  on  the  PIC32  while  performing  this  grasp.  It  indicates  that  the  microcontroller 
completed  the  computation  in  approximately  12ms.  The  rest  is  handed  over  to  the  I/O  blocks  of  the  microcontroller 
and  to  the  motor  actuation  stage. 


Figure  11.  Computational  load  on  the  PIC32  during  the  grasp. 

These  results  suggest  an  extensible  design  for  increased  load  on  the  processing  by  the  embedded  system. 


Real-Time  Operating  System  for  Embedded  Control 

For  our  work  FreeRTOS  is  used  to  schedule  several  tasks  that  the  microcontroller  needs  to  do,  based  on  their 
priority  levels.  These  tasks  include  sEMG  analysis,  high-level  control  (i.e.  the  control  strategy),  low-level  control 
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(i.e.  the  motor  control),  receive  data  from  contact  sensors,  receive  data  from  force  sensors,  and  receive  data  from 
encoders,  among  others.  All  these  tasks  need  not  be  done  at  the  same  frequency.  Some  of  them  might  happen  only 
occasionally.  In  case  of  contact  an  interrupt  might  happen  and  the  controller  algorithm  might  change,  based  on  the 
event  driven  structure.  Control  load  for  each  situation  might  be  different.  A  task  is  an  independent  thread  of 
execution  that  contains  a  sequence  of  independently  schedulable  instructions.  Multitasking  is  the  ability  of  the 
operating  system  to  handle  multiple  activities  within  set  deadlines.  The  Free  RTOS  multitasks  in  such  a  way  that 
many  threads  of  execution  appear  to  be  running  concurrently;  however,  the  OS  is  actually  interleaving  executions 
sequentially,  based  on  a  preset  scheduling  algorithm.  The  scheduler  ensures  that  the  appropriate  task  runs  at  the  right 
time.  FreeRTOS  supports  up  to  256  priority  levels,  in  which  0  is  the  highest  and  255  the  lowest.  With  a  preemptive 
priority-based  scheduler,  each  task  has  a  priority,  and  the  highest-priority  task  runs  first.  If  a  task  with  a  priority 
higher  than  the  current  task  becomes  ready  to  run,  the  kernel  immediately  saves  the  current  task's  context  and 
switches  to  the  higher-priority  task.  When  that  is  done,  the  control  is  returned  to  the  original  task. 

An  experimental  setup  similar  to  the  one  shown  in  Figure  3,  but  with  Free  RTOS  installed  on  the  PIC32-I,  was  built. 
For  proof  of  concept,  the  embedded  processor  running  a  Free  RTOS  kernel  was  connected  to  two  different  sets  of 
components:  a  DC  motor  and  a  group  of  LED's.  An  application  program  was  built  and  released  onto  the  PIC32-I, 
which  drove  the  DC  motor  and  flashed  the  LED's  by  way  of  interleaving  the  two  separate  tasks,  with  equal  priority 
levels  assigned  to  both  tasks.  The  application  performed  as  designed  without  any  noticeable  delay.  While  the 
computational  loads  of  these  tasks  are  insignificant,  this  experiment  demonstrated  the  value  of  Free  RTOS  and  its 
potential  use  at  least  in  the  low-level  control,  where  multiple  motors  or  sensing  devices  will  likely  be  employed. 

External  Mass  Storage  o  f  On-line  EMG  Data 

Because  embedded  processors  such  as  the  PIC  have  limited  on-chip  storage,  interfacing  with  external  devices  helps 
increase  the  data  managing  capability  for  our  embedded  control.  In  order  to  use  these  external  devices  such  as  the 
Secure  Digital  (SD)  memory  cards,  we  need  to  initialize  the  device.  We  tested  a  simple  data  sending  and  receiving 
pair  as  follows.  For  reading  data  from  the  SD  card  initially  we  sent  a  Read_Signal  Command,  then  wait  for  the  SD 
card  to  respond  with  a  specific  token  DATA_START.  Once  the  token  is  received  we  can  confidently  read  in  a  rapid 
sequence  all  512  bytes  composing  the  requested  block  of  data.  This  was  followed  by  16-bit  Cyclic  Redundancy 
Check  (CRC)  and  finally  we  can  retrieve  data  from  device.  Data  sending  worked  as  follows:  Send  a  Write  Signal 
command  and  check  the  SD  card's  response  to  make  sure  that  the  command  is  accepted.  Send  DATASTART  token 
and  immediately  after  it,  in  a  short  loop,  all  512  bytes  of  data.  Send  2  bytes  for  the  16-bit  CRC  (with  any  dummy 
value).  The  token  DATA_ACCEPT  will  confirm  that  the  entire  block  of  data  has  been  received  and  the  write 
operation  has  started.  Wait  for  the  completion  of  the  write  command.  While  the  card  is  busy  writing,  it  will  keep  the 
SDO  (Send  Data  Out)  line  low.  So  we  will  wait  for  the  SDO  line  to  return  high.  We  need  to  take  care  of  the 
interrupts  and  communication  path  from  the  external  device  to  the  PIC  to  ensure  both  read  and  write  operations  are 
completed  successfully. 

Advance  in  technology  helped  to  include  a  USB  interface  on  the  PIC32-II.  With  this  we  can  directly  accesses  the 
USB  for  reading  and  writing  the  data  from  and  to  the  USB.  The  USB  module  consists  of  the  Clock  Generator,  USB 
Voltage  Comparator,  Transreceiver  and  SIE  Dedicated  DMA  controller.  The  PIC32-II  also  includes  a 
PIC32MX440F512H  USB  micro  controller  that  provides  debugged  connectivity  over  USB.  It  provides  2  types  of 
protocol  translation  debugging  USB  connectivity.  The  USB  peripheral  stack  consists  of  the  following  layers: 

Application  Layer-  Consists  of  the  firmware  needed  to  implement  the  device  desired  behavior. 

Function  Drivers-  Function  drivers  implement  the  desired  function  behavior  and  provide  function-specific  control 
interfaces  to  the  application.  To  access  the  USB  and  transfer  data  and  control  information,  function  drivers  interact 
with  the  USB  device  layer. 

USB  Device  Layer-  Handles  the  protocol  elements  specified  by  the  manufacturer.  The  device  layer  communicates 
with  the  Hardware  Abstraction  Layer  as  necessary  to  support  interface. 

Hardware  Abstraction  Layer  (HAL)-  Abstracts  the  USB  controller  hardware  and  provides  accesses  to  all  the 
features  that  the  controller  implement  to  support  a  USB  peripheral  Device. 
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We  began  preliminary  work  to  build  a  look-up  table  of  sEMG  data  based  on  the  entropy  corresponding  to  the  ring, 
middle  and  index  fingers.  The  next  step  is  to  implement  a  table  and  store  it  on  USB-based  storage  devices,  and 
programming  the  processor  to  read  the  data  from  USB  and  store  the  sEMG  data  during  on-line  acquisition  of  sEMG 
data. 


Modeling  and  Simulation  of  the  Embedded  Control 

One  of  the  aspects  that  should  be  considered  in  this  project  is  modeling  and  simulation  of  prosthetic  hand.  The 
simulation  provides  an  environment  to  test  the  functionality  of  hand  and  evaluating  performance  of  different  control 
algorithms.  A  simulation  environment  can  also  provide  visualization  of  hand  which  can  be  used  in  training  the 
amputees  before  they  use  the  actual  prosthetic  hand.  A  system  design  which  has  been  used  by  Johns  Hopkins 
University  in  order  to  simulate  a  prosthetic  hand  in  an  integrated  environment  is  studied  and  the  features  of  used 
elements  is  described  and  discussed. 


Virtual  Integrated  Environment 

The  Virtual  Integrated  Environment  (VIE)  is  a  simulation  system  to  model  prosthetic  hand  as  close  as  possible  to  a 
real  hand  in  a  virtual  environment.  This  system  is  developed  by  Johns  Hopkins  University  and  it  is  a  combination  of 
different  software  which  communicates  with  each  other.  The  main  elements  of  the  system  include:  dynamic 
modeling,  visualization,  EMG  data  acquisition,  and  control.  If  it  is  required  to  model  the  prosthetic  hand  as  a  part  of 
body,  some  optional  parts  might  be  added  to  the  system.  In  this  case  it  is  necessary  to  track  body  movements,  and  an 
optical  tracking  system  handles  this  part.  The  structure  of  VIE  is  shown  in  Figure  12. 
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Figure  12.  Block  diagram  of  the  VIE  elements 


Because  the  system  is  made  of  different  software  and  elements,  the  TCP/IP  and  UDP/IP  standard  protocols  are  used 
to  provide  communication  between  these  parts  at  the  software  level;  and  RS232  is  used  at  hardware  level  for  similar 
purposes.  The  VIE  is  capable  of  testing  the  functionality  of  the  prosthetic  hand  and  training  amputees.  However,  the 
necessity  of  simulation  is  not  limited  to  the  above  mentioned  purpose  and  in  design  stage  each  design  team  might 
need  to  use  some  sort  of  simulation.  One  of  the  main  simulation  needs  is  dynamic  modeling  and  visualization.  The 
MSMS  software  is  used  for  this  purpose. 


MSMS  (Musculoskeletal  Modeling  Software ) 

MSMS  is  a  software  application  for  modeling  and  simulation  of  neural  prostheses  systems.  It  can  be  used  to  model 
and  simulate  human  and  prosthetic  limbs,  and  the  task  environment  they  operate  in.  The  simulations  can  be  executed 
in  a  standalone  computer  to  develop  and  test  neural  control  systems  or  in  a  virtual  reality  environment  where  the 
human  or  animal  subject  can  interact  with  and  therefore  affect  the  behavior  of  the  simulated  limb.  MSMS  is  based 
on  standard  XML  format  for  model  description.  It  has  some  limited  tools  for  creation,  editing,  and  visualization  of 
3D  models.  To  use  more  complicated  models  it  has  the  ability  to  import  models  from  SIMM  and  SolidWorks. 
Ability  to  model  human  limbs  and  prosthetic  limbs  and  the  task  environments  is  a  main  feature  of  MSMS.  The  input 
commands  to  the  hand  can  be  stored  in  motion  files  or  can  be  streamed  in  real-time  from  a  simulation  or  motion 
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capture  system.  It  also  has  the  ability  to  playback  animation  sequences.  The  prosthetic  hand  can  interact  with  the 
environment  and  finally  the  software  can  be  used  for  virtual  environment  and  subject  in  the  loop  simulation  of 
prosthetic  hand.  The  solver  used  in  the  software  is  SimMechanics  which  is  part  of  the  Matlab/Simulink  software. 
SimMechanics  is  a  multi-body  mechanical  simulation  system.  In  this  software  instead  of  driving  dynamic  equation 
mechanical  elements  like  joints,  constraints  and  force  elements  are  attached  together  and  modeled  as  a  system. 
MSMS  provides  a  tree  structure  to  make  the  model,  which  means  each  part  can  include  some  subparts.  This  feature 
makes  the  model  easier  to  access  and  edit  for  later  use,  and  it  is  based  on  XML  description  of  the  model.  The 
structure  of  the  software  considering  importing  the  CAD  model  from  SolidWorks  is  shown  in  Figure  13. 


Figure  13.  Importing  models  from  SolidWorks  to  MSMS  and  from  MSMS  to  Simulink 

SolidWorks  outputs  a  Physical  Modeling  XML  file  describing  the  mechanical  linkage  and  STL  files  representing  the 
appearance  of  each  segment.  Both  sets  of  outputs  are  used  by  MSMS  to  create  an  accurate  model  of  the  prosthetic 
limb  in  MSMS.  The  final  Simulink  model  can  be  built  via  Matlab/SimMechanics  or  MSMS.  But  the  use  of  MSMS 
allows  the  users  to  attach  the  imported  prosthetic  limb  to  a  human,  add  additional  prosthetic  components  such  as 
actuators,  and  simulate  it  in  an  appropriate  task  environment. 

Simulation  Mode 

MSMS  models  can  be  automatically  converted  into  Simulink  for  physics-based  simulations  of  the  model4  s  behavior 
in  response  to  control  inputs  and  external  forces.  By  defining  the  inputs  and  outputs,  MSMS  model  converts  as  a 
block  into  the  Simulink  environment.  By  use  of  xPC  target  it  is  possible  to  run  the  model  in  real  time  and  for  virtual 
reality  simulation  and  subject  in  the  loop  purposes.  Figure  14  depicts  this  conversion  process  from  MSMS  to 
Simulink  and  xPC. 
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Automatic  Conversion 


Figure  14.  Model  conversion  from  MSMS  to  Simulink  and  xPC  target  for  real-time  simulation 


Solver  Performance 

As  mentioned  above,  the  solver  uses  SimMechanics.  So  the  solver  features  are  similar  to  Matlab/Simulink 
environment.  The  initial  time,  final  time,  and  step  size  can  be  set;  and  simulation  runs  based  on  these  parameters. 
The  smaller  steps  means  more  accurate  simulation  which  is  necessary  especially  when  the  subject  is  interacting  with 
the  environment,  and  when  modeling  collision  and  grasping.  On  the  other  hand,  the  computation  load  increases  and 
running  simulation  in  real  time  might  be  impossible  as  the  steps  become  very  small  -  this  suggests  a  computational 
performance  bottleneck. 


Page  128  of  531 


Goal  3:  Robotic  Hand 


We  proposed  two  deliverables  within  Goal  3  for  Phase  II,  which  can  be  divided  in  three  objectives.  The  first  one  is 
the  implementation  and  testing  of  the  hand  prototype  developed  in  Phase  I  (Prototype  1),  whose  main  characteristic 
is  the  parallel  actuation  system.  The  second  one  is  the  design  of  the  new  prototype  (Prototype  2),  combining  the 
results  of  the  testing  with  newly  developed  kinematic  synthesis  results,  and  with  the  results  from  the  EMG  signal 
identification.  The  third  goal  is  to  explore  the  manipulation  of  grasped  objects  from  a  theoretical  point  of  view. 
Below  find  the  detailed  report  of  work  and  findings  for  each  of  these  objectives. 

Goal  3,  Objective  1:  Implementation  and  testing  of  hand  Prototype  1 

The  outcome  of  the  design  from  Phase  I  consisted  of  a  two -finger  plus  thumb  prototype  with  parallel  actuation  for 
mimicking  the  actions  of  the  two  main  finger-flexing  muscles,  FDP  and  FDS  (see  final  Report  of  Phase  I).  From 
now  on,  we  will  call  this  design  Prototype  1 . 

Prototype  1  function 

According  to  the  results  of  the  testing  of  the  parallel  actuation,  and  to  the  current  findings  of  our  Goal  1:  EMG 
Signal  Identification  and  Estimation,  Prototype  1  will  keep  the  parallel  actuation  structure,  but  its  purpose  has  been 
modified:  the  parallel  actuation  system  is  to  be  used  to  drive  separately  the  signals  from  position  and  force.  The 
light,  not-so-strong  actuators  at  each  finger  limb  will  be  driven  by  the  motion,  while  this  will  be  complemented,  if 
needed,  by  the  stronger,  coupled  actuators  at  the  palm  when  more  force  is  needed. 

The  manufacturing  of  Prototype  1  has  been  continued  and  extended  for  a  full  hand,  with  slight  design  modifications 
to  correct  flaws  that  were  identified  during  the  initial  testing.  The  final  version  of  Prototype  1  has  the  parallel 
actuation  system  and  position  /  force  feedback  sensing. 


Figure  1:  Hand  Prototype  1  design 

The  refinement  of  the  Phase  I  prototype  has  led  to  a  five -fingered  anthropomorphic  hand  (Figure  1)  based  on  the 
design  carried  out  at  Phase  I,  has  been  manufactured  and  assembled  with  the  electrical  actuation  structure  (Figure  2). 
There  has  been  some  redesign  for  a  better  functionality  and  ease  of  assembly. 
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Figure  2:  The  assembled  hand  Prototype  1 


Electrical  actuators 

The  muscles  of  most  interest  for  this  research  and  most  responsible  for  flexural  grip  of  the  finger  are  the  FDS  and 
FDP  muscles.  The  FDP  muscle  reaches  its  maximum  force  exertion  at  approximately  35%  of  the  total  flexural 
effort;  however,  the  FDS  muscle  continues  to  exert  force  until  it  reaches  about  100%  of  the  total  flexural  effort.  It  is 
to  be  noted  that  the  actual  act  of  gripping  requires  much  less  complex  finger  motion  than  that  of  manipulation. 
Based  on  these  observations,  the  FDS  and  FDP  strength  space  was  divided  into  the  region  1,  which  represents  the 
more  frequent  dexterous  tasks,  and  region  2  i.e.  the  less  frequent  and  more  strength  based  tasks.  Having  this  concept 
the  design  of  the  actuation  has  been  implemented  accordingly. 

The  region-one  actuation  scheme  for  the  fingers,  i.e.  DIP/PIP  DC  motors  and  the  MCP  DC  motors,  has  been 
assembled  and  tested  for  different  grasping  configurations.  Figure  3  shows  the  actuators  within  the  hand.  The 
actuators  used  in  the  prototype  are  Pololu  35:1  Mini  Metal  Gearmotors.  The  DC  motors  are  coupled  to  the  joints  of 
the  finger  via  two  bevel  gears. 


Figure  3:  Region-1  actuation  scheme  for  finger 


Experimental  setup  and  Testing 
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The  basic  requirements  for  a  prosthetic  hand  are  grasping  functionality  and  simple  controllability.  These  different 
requirements  cannot  be  addressed  separately  because  they  are  strictly  connected  and  can  be  faced  as  different 
issues  of  one  fundamental  challenge  to  develop  appropriate  and  suitable  artificial  hands  able  to  ultimately 
replicate  the  characteristics  and  performance  the  human  hand. 

In  order  to  assess  whether  a  design  is  successful  from  a  functional  point  of  view,  a  well-defined  set  of  tasks  and 
performance  indices  are  needed,  see  (Ellul  and  Saliba  2010).  The  development  of  the  standards  and  benchmark  tests 
for  Prototypes  1  and  2  is  a  task  that  is  currently  being  developed  and  we  expect  to  finish  in  the  next  three  months. 

In  parallel,  some  testing  has  been  performed  in  order  to  assess  basic  functional  aspects  of  Prototype  1  as  well  as  its 
basic  interaction  with  the  EMG  driving  signals.  This  testing  is  described  below  and  is  partially  covered  in  one  of  our 
publications  (Crawford,  et  al.  2010). 

Testing  for  Grasping  Performance 

The  grasping  ability  and  the  reaching  of  each  finger  have  been  analyzed  in  the  prototype.  Fingers-thumb 
opposability  enables  the  hand  to  carry  out  prehensile  movements  of  the  fingers  to  hold  an  object  securely.  Here  we 
focus  on  two  main  patterns  in  prehensile  grasping,  the  precision  grip  (or  grasp),  between  the  terminal  digital  pad  of 
the  opposed  thumb  and  the  pads  of  the  fingertips,  and  the  power  grip  (or  grasp). 

Holding  large  objects  involves  all  the  digits,  while  small  objects  require  only  the  thumb,  the  index,  and  the  middle 
digits.  Smaller  objects  may  be  pinched  specifically  between  the  thumb  and  index  (or  middle)  finger,  which  should 
be  adapted  to  perform  fine  control.  The  power  grip  is  executed  between  the  surface  of  the  fingers  and  the  palm, 
which  acts  as  a  buttressing  and  reinforcing  agent.  All  these  grasping  actions  have  been  visually  checked  in  the 
prototype;  see  Figure  4  as  an  example. 


Figure  4:  Prototype  hand  holding  a  spherical  object. 


Experiments  on  Force  Control 

In  order  to  estimate  the  force  exerted  at  the  grasping,  a  system  based  on  FSR  has  been  designed.  The  experiment  has 
been  conducted  as  follow. 

The  two  pololu  35:1  mini  metal  gear  motors  in  the  finger  are  driven  by  PWM  which  is  generated  by  the 
microcontroller. 

The  FSR  is  attached  to  the  tip  of  the  finger  or  to  the  ball  that  we  wanted  to  grasp  and  for  feedback  it  is 
connected  back  to  the  micro  controller  (Figure  5). 

The  motors  keep  on  rotating  until  the  finger  is  in  contact  with  the  object;  however,  when  it  keeps  on  pressing 
further,  the  motors  are  commanded  to  rotate  back  and  stop  in  contact  position.  It  is  a  promising  simple  result  that 
could  be  part  of  a  strategy  for  the  amputee  to  grasp  different  objects  like,  glass,  egg,  etc,  with  some  control  on  the 
exerted  force  depending  on  the  fragility  of  the  object.  Results  and  details  of  the  control  mechanism  and  algorithm 
will  be  further  discussed  in  the  control  and  embedded  systems  section. 
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Figure  5:  Contact  position  control  experiment  setup 


Actuation  Experiment  with  simple  EMG  Controller 

In  this  experiment,  EMG  signal  capturing  capability  was  added  to  control  the  DC  motors.  The  raw  signals  were 
processed  by  taking  the  maximum  absolute  value  over  a  100  sample  interval  and  generating  a  new  data  set.  In  order 
to  get  the  full  flex  and  extend  ability  of  the  finger  from  this  new  data  set  two  threshold  levels  where  introduced. 
Above  one  threshold  the  motor  would  spin  in  one  direction  (flex)  and  below  another  the  motor  would  spin  in  the 
opposite  direction  (extend).  The  deadzone  was  implemented  as  a  buffer  area  to  transition  between  the  flexion  and 
extension  zones. 

The  EMG  signal  was  acquired  from  the  digit  III  FDP  muscle  and  supplied  as  an  analog  voltage  from  a  separate  PC 
incorporating  a  Delsys  Bagnoli  EMG  system.  The  analog  voltage  was  sampled  through  a  National  Instruments  data 
acquisition  card  installed  in  the  motor  control  PC.  The  analog  signal  was  then  filtered  and  scaled  using  Lab  VIEW  to 
a  value  representing  the  amplitude  of  the  EMG  signal,  which  was  then  used  to  control  the  direction  of  the  motors.  A 
sample  of  the  filtered  and  scaled  signal  with  thresholds  is  shown  in  Figure  6. 


The  controller  was  applied  to  the  DC  motors  of  finger  and  thumb  only,  in  their  close  grip  configurations.  The 
objective  of  these  tests  is  to  obtain  some  preliminary  data  in  order  to  setup  the  EMG -testing  platform. 
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The  EMG  voltage  and  current  outputs  for  both  the  finger  and  thumb  were  consistent  with  that  of  the  direct  power 
supply  inputs.  The  EMG  thumb  test  resulted  in  a  DC  motor  force  of  0.95N  which  agreed  with  the  1.24N  +  0.35N 
generated  by  the  direct  power  supply  tests  presented  in  the  Final  Report  of  Phase  I. 

Prototype  1  -  Redesign  of  key  parts 

While  testing  the  prototype,  problems  were  identified  and  solutions  are  being  developed.  For  some  of  them 
measures  have  already  been  taken.  It  is  important  to  notice  that  this  is  an  ongoing  process,  as  more  testing  leads  to 
the  identification  of  further  problems.  Some  of  these  will  be  implemented  in  Prototype  1,  while  some  will  be 
implemented  in  Prototype  2  once  this  is  in  a  more  mature  stage. 

The  palm  and  fingers  attachment  point  has  been  modified  in  such  a  way  that  the  assembly  is  easier,  while  keeping  its 
function  in  providing  a  passive  DOF  for  the  abduction  of  the  fingers  (Figure  7). 


joint 


The  new  modification 


Figure  7:  Knuckle  jointed  with  the  palm 


As  shown  in  the  figure,  in  the  first  case  the  part  was  designed  so  that  it  will  be  attached  by  a  screw,  but  as  the  part  is 
loaded  with  torsion  and  bending  moments  the  screw  becomes  loose;  we  also  observed  interference  with  the  gears. 
We  found  this  new  approach  is  a  good  solution  to  keep  the  part  in  place  and  still  provide  access  for  maintenance  of 
the  gear  pin. 

Modification  on  the  thumb,  particularly  on  the  actuator  of  the  CMC  joint  has  been  also  carried  out.  In  the  model,  one 
of  the  CMC  joint  of  the  thumb  (flexion  motion)  was  designed  so  that  it  could  be  fully  actuated  by  the  SMA. 
However,  due  to  the  orientation  of  the  thumb  and  due  to  the  substantial  hysteresis  (due  to  energy  dissipation)  that 
could  occur  on  the  heating  and  cooling  cycles,  accurate  control  of  SMAs  could  be  difficult  and  hence  variation  of 
force  in  the  thumb  is  expected.  The  thumb  must  generate  much  higher  forces  than  any  other  finger  when  performing 
certain  gripping  actions  due  to  the  thumb  opposing  the  total  force  generated  by  the  four  fingers.  This  thumb  behavior 
is  prevalent  in  common  grip  types  such  as  a  pinch  grip  and  tripod  grip.  Therefore  an  alternative  actuator  is  currently 
being  characterized  for  this  application. 

In  order  to  perform  further  experiments,  and  in  order  to  drive  the  flexion  motion  of  the  CMC  joint,  the  muscle-like 
string  with  a  DC  motor  is  implemented.  This  is  the  focus  of  ongoing  work  and  on  completion  will  be  integrated  with 
the  current  design.  However,  we  will  also  implement  the  parallel  actuation  technique  using  SMA  or  similar  actuators 
as  it  is  designed  to  provide  additional  power  to  the  other  fingers,  too. 


Prototype  1  -  Working  principle  of  the  newly  proposed  actuation  scheme. 

Here  the  approach  is  similar  to  SMA  actuation  mechanism  designed  before,  however  because  of  the  above  and  some 
other  reasons,  we  are  experimenting  with  the  use  of  DC  motors  and  a  belt  system  to  actuate  the  specified  joint.  A  pin 
is  fixed  with  the  thumb  metacarpal  and  a  muscle  like  string  will  be  fixed  to  it,  this  string  will  be  wind  up  on  a  wheel 
driven  by  a  DC  motor.  To  return  to  its  original  position  there  is  a  torsion  spring  mounted  in  the  joint  as  shown  in  the 
Figure  8. 
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Figure  8:  Actuation  scheme  for  the  CMC  thumb  joint 


Summarizing  the  tasks  for  the  testing,  basic  testing  and  redesign  are  being  performed  on  Prototype  1 .  At  the  same 
time,  a  set  of  standards  for  testing  is  to  be  developed.  We  expect  to  have  the  final  working  version  of  Prototype  1 
and  the  testing  standards  and  testing  results  finalized  within  3  months. 

Goal  3,  Objective  2:  Design  of  hand  Prototype  2 

In  parallel  to  the  tasks  being  realized  for  Objective  1,  and  based  on  our  experience  with  Prototype  1,  we  are  working 
on  developing  Prototype  2.  While  conducting  the  experiment  on  Prototype  1,  some  problems  were  identified  that 
will  be  considered  in  Prototype  2.  To  mention  some  of  the  issues,  we  list  the  noise  coming  from  the  gears,  backlash 
problem  in  the  gears  and  as  the  design  was  targeted  in  using  parallel  actuation,  power  supplied  by  the  DC  motor 
alone  was  not  sufficient  to  hold  things  firmly.  Some  of  the  parts  still  need  further  modification,  as  it  is  not 
convenient  for  the  artificial  hand  to  be  assembled  and  disassembled  with  relative  ease.  This  will  be  a  disadvantage 
for  the  end  user,  as  these  features  of  the  design  can  only  lead  to  enhanced  user  confidence  in  the  prosthesis  solution. 
Therefore,  keeping  the  current  prototype  as  a  test  bed  for  the  control,  EMG  and  interfacing  experiments,  further 
improvements  on  those  problems  and  other  prosthetic  issues  will  be  addressed  in  the  new  design  for  Prototype  2. 

For  this  prototype,  we  are  focusing  on  the  design  of  the  palm  and  thumb,  the  exploration  of  new  actuation  systems, 
and  the  use  of  kinematic  synthesis  in  order  to  obtain  novel  designs. 


Literature  Review 

The  research  on  the  state  of  the  art  in  a  field  as  dynamic  as  this  one  needs  to  be  revisited  on  a  short  basis.  Below  we 
present  the  review  performed  during  this  first  year  of  Phase  II. 

In  the  development  of  an  appropriate  multifunctional  prosthesis,  one  must  balance  enhanced  performance  with  the 
consequent  increases  in  cost  and  complexity  in  the  finding  of  more  EMG  channels  that  it  may  incur.  However,  the 
prosthetic  hand  still  has  to  provide  useful  levels  of  force  and  appropriate  speeds  of  motion  relative  to  the  native  hand 
and  possesses  an  anthropomorphic  skeletal  structure  to  promote  natural  function  and  appearance. 

Several  prosthetic  hands  have  recently  been  described  in  engineering  research  papers  (Dailey  and  Wiste  2009)  - 
(Caldwell  and  Tsagarakis  2002).  In  each  device,  the  discrepancy  between  the  number  of  independent  actuators  and 
the  number  of  joints  is  accommodated  either  by  differential  drives,  which  prescribe  a  given  torque  distribution 
between  joints,  kinematic  linkages,  which  prescribe  a  given  relative  motion  between  joints,  variable  compliance 
couplings,  which  prescribe  given  relative  compliance  between  joints,  or  by  a  combination  of  these. 
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Review  of  Kinematical  Structures 

Researchers  have  approached  in  different  ways  the  design  in  order  to  mimic  the  human  hand,  using  conventional 
linkages  and  other  mechanical  transmission  systems.  The  prosthetic  hand  in  (Dailey  and  Wiste  2009)  is  designed  to 
provide  selected  functions  with  a  less  number  of  independent  control  inputs.  It  possesses  five  independent  actuators 
and  16  joints,  hence  it  is  an  underactuated  design.  The  discrepancy  between  the  number  of  joints  and  actuators  is 
accommodated  by  moment  isotropy,  wherein  each  digit  achieves  static  equilibrium  when  the  moment  in  each  joint  is 
essentially  equal  to  the  other  joints  in  the  same  actuation  chain.  Moment  isotropy  is  realized  by  a  combination  of 
tendons  spanning  multiple  joints  and  a  differential  pulley  mechanism  that  distributes  forces  across  tendons.  In  each 
joint  spring  has  been  used  for  compliance  by  then  eliminate  the  need  for  position  sensing,  as  the  compliant  joint  map 
the  finger  positions  to  the  tendon  force.  Back  drivable  force  controlled  actuation  system  has  been  also  implemented 
so  that  the  user  can  have  a  sense  of  feeling  of  the  force.  The  muscle  contraction  sensed  by  the  user  concurrently 
generates  the  input  for  the  prosthetic  device  through  EMG  signals.  Backdriveablity  then  allows  for  a  proportional 
mapping  of  this  input  to  device  output.  Each  fingers  has  three  DOF,  the  thumb  has  one  additional  DOF  for 
opposition.  See  Figure  9 


Figure  9:  Sixteen-joint  anthropomorphic  hand 


In  (Oshima  and  Takayama  2009)  a  modular  hand  that  can  be  inserted  through  trocars  for  robotic  hand  assisted 
laparoscopic  surgery  (HAFS)  has  been  proposed.  The  robot  Hals  has  three  fingers  and  nine  active  degrees  of 
freedom  and  three  passive  DOF.  Here  tendons  are  also  used  for  power  transmission  and  as  the  design  is  targeted  for 
manipulations  these  three  fingers  hold  different  configurations  due  to  the  middle  finger  has  the  freedom  to  translate 
forward  and  backward. 
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Right  finger 


[mm) 


Left  Finger 


Center  finger 


Figure  10:  Assembled  three-fingered  nine-dof  hand 


Designed  by  the  Robotic  Systems  Technology  Branch  at  the  NASA  Johnson  Space  Center  to  reproduce  the  size, 
kinematics  and  strength  of  the  space  suited  astronaut  hand  and  wrist,  the  Robonaut  Hand  (Lovchik  1999)  has  an 
anthropomorphic  configuration  with  5  fingers  and  twelve  degrees  of  freedom.  The  wrist  has  two  DOF  for  a  total  of 
fourteen  degrees  of  freedom.  A  forearm  completes  the  structure,  housing  all  fourteen  motors,  and  other  circuit 
boards.  In  order  to  match  the  size  of  an  astronaut4 s  gloved  hand,  the  motors  are  mounted  outside  the  hand,  and 
mechanical  power  is  transmitted  through  a  flexible  drive  train.  To  avoid  reliability  problems  associated  with  tendons 
when  used  in  EVA  space  environment,  the  hand  uses  flex  shafts  to  transmit  power  from  the  motors  in  the  forearm  to 
the  fingers.  The  rotary  motion  of  the  flex  shafts  is  converted  into  linear  motion  in  the  hand  using  small  modular 
leadscrew  assemblies  resulting  in  a  compact  drive  train,  see  Figure  11. 


Linkages 


Figure  11:  Grasping  finger  and  decoupling  link  of  the  Robonaut  Hand 


Developed  by  the  Shadow  Robot  Company,  the  Shadow  Hand  (Shadow  Robot  Company  n.d.)  has  24  degrees  of 
freedom,  distributed  as  follows:  4  DOF  each  for  the  first,  middle  and  ring  finger,  5  DOF  for  the  little  finger,  5  DOF 
for  the  thumb  and  2  DOF  for  the  wrist.  The  shadow  hand  utilizes  40  air  muscles  to  actuate  its  24  degrees  of  freedom 
in  a  volume  resembling  a  very  large  forearm.  In  terms  of  speed,  the  Shadow  Hand  has  a  general  movement  at 
approximately  half  the  speed  of  a  human  hand.  Though  the  shadow  hand  looks  bulky  it  has  overall  weight  of 
approximately  4  kg  only  including  sensors,  muscles  and  valve  manifold,  due  to  the  materials  used  to  build  it.  Steel  is 
only  used  for  the  forearm  bone  while  other  materials  as  aluminum,  polycarbonate,  rubber,  etc  are  used  to  built 
fingers,  palm  and  base.  See  Error!  Reference  source  not  found.Figure  12  and  Error!  Reference  source  not 
found.Figure  13. 


Page  136  of  531 


Figure  12:  The  Shadow  Dexterous  Robot  Hand 


Figure  13:  Configuration  of  joints  for  the  Shadow  Hand 


Review  of  Sensing  Technologies 

Given  the  complexity  of  the  human  hand  sensory  system,  the  current  technology  is  not  enough  to  satisfy  the 
prosthetic  users.  With  some  limitations,  several  sensor  technologies  have  been  applied  to  position  and  tactile 
sensing.  One  common  technology  is  the  force-sensing  resistor  (FSR),  which  are  made  of  a  resistive  polymer  film 
that  responds  in  a  non-linear  way  to  pressure.  Another  technology  is  quantum-tunneling  composite  (QTC).  When 
deformed,  QTC  material  turns  from  a  good  insulator  into  a  conductive  material.  Resistance  drops  exponentially  as  a 
function  of  pressure.  Other  force-sensing  technologies  are  based  on  semiconductor  strain  gauges.  These  have  been 
used  to  measure  pressure  and  to  create  multi -dimensional  load  cells.  Multi-dimensional  load  cells  are  particularly 
good  for  tactile  sensing  because  they  can  be  very  sensitive  and  they  can  be  used  to  localize  a  contact  point  on  a 
fingertip.  However,  a  drawback  to  using  a  load  cell  to  sense  forces  in  a  robot  hand  has  been  size  constraints. 

In  (Dailey  and  Wiste  2009),  the  user‘s  musculature  (from  which  the  myoelectric  command  signals  are  measured) 
possesses  force-sensing  capabilities.  By  incorporating  force -controlled,  backdrivable  actuation,  some  degree  of  force 
sensing  can  be  reserved  (i.e.,  the  tendon  forces  retain  some  degree  of  proportionality  with  the  EMG  command).  It 
was  assumed  (and  is  the  case  with  myoelectric  devices)  that  no  conduit  exists  to  bring  sensory  information  to  the 
user.  Because  of  this,  the  device  was  designed  to  be  force -controlled,  such  that  the  information  contained  in  the 
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muscle  contraction  is,  in  essence,  preserved  in  the  prosthesis.  Encoders  are  implemented  to  sense  the  motor  position. 

Prosthetic  hands  contain  a  sensory  feedback  device,  for  instance  the  Belgrade/USC  Hand  (Bekey,  Tomovic  and 
Zeljkovic  1990)  has  small  conductive  plastic  potentiometers  compact  (35x1 0x3 mm)  with  good  resolution 
(320ohms/mm)  of  position  sensors  to  indicate  the  rotation  of  the  fingers  base  with  respect  to  palm  plane.  The  same 
hand  also  contains  FSR  located  at  the  finger  tips  to  detect  contact  with  an  object  and  the  force  being  exerted  in  the 
range  of  20  grams  to  5  kg.  These  sensors  can  also  report  contact  events  when  they  sense  a  minimum  normal  force. 
Many  hands,  such  as  the  Utah/MIT  hand,  contain  internal  sensors  located  at  each  joint  to  measure  angular  deflection 
and  the  sensor  system  is  based  on  a  Hall  Effect.  A  magnetically  sensitive  Hall  Effect  device  is  located  in  the 
proximal  link  and  two  cobalt  samarium  magnets,  operating  in  a  dipole  configuration,  are  attached  to  the  distal  link. 
Also  semiconductor  strain  gage  to  detect  the  tendon  tension.  The  Shadow  Hand  also  contains  a  similar  Hall  Effect 
position  sensor  with  resolution  of  0.2  degrees. 

The  Robonaut  hand  contains  forty  three  sensors  not  including  tactile  sensing.  Each  joint  is  equipped  with  an 
embedded  absolute  position  sensor.  Each  motor  is  equipped  with  absolute  encoders.  Each  of  the  lead  screw 
assemblies  as  well  as  the  wrist  ball  joint  links  are  instrumented  as  load  cells  to  provide  force  feedback. 

Usually  if  we  look  for  a  prosthetic  hand  to  perform  a  dexterous  task  we  need  a  feedback  from  each  joint  to  control 
the  particular  actuators,  however  while  doing  this  the  control  complexity  is  also  increased.  Because  of  these  and 
other  reasons  most  of  the  prosthetic  hands  developed  so  far  are  limited  to  power  grasping  and  use  a  more  pragmatic 
and  robust  underactuated  approach. 

Review  on  Actuator  Systems 

The  actuation  units  in  (Dailey  and  Wiste  2009)  are  DC  motors  coupled  with  low-ratio  gearheads  and  small-diameter 
pulleys  to  pull  hand  tendons.  Each  joint  in  the  hand  includes  embedded  torsional  springs  in  parallel  with  the  hand 
tendons.  The  compliant  joints  map  joint  motion  to  tendon  force  in  free  space  gives  advantages  of  1)  eliminating  the 
need  for  position  sensing  in  the  hand,  and  2)  eliminating  the  need  to  switch  between  motion  control  and  force 
control.  Specifically,  the  hand  frequently  engages  in  both  motion  control  (e.g.,  when  gesturing  or  reaching)  and 
force  control  (e.g.,  when  grasping  or  squeezing).  In  the  hand,  the  tendons  are  always  under  (open-loop)  force  control 

Tendon  paths  are  lined  with  Teflon  tubing  to  reduce  friction.  Braided  spectra  cable  of  0.75mm(0.03  in)  diameter  and 
668  N  (150  lbf)  rated  strength  was  chosen  for  tendon  material  due  to  its  high  strength,  high-tensile  fatigue 
resistance,  low  creep,  and  low  stretch  characteristics. 

Progress  in  materials  science,  microcontrollers,  batteries  and  cosmoses  products  made  feasible  the  design  of 
advanced  prosthetic  devices.  However,  due  to  the  complexity  and  higher  DOF  of  the  hand  prostheses  have  still 
restricted  functions,  mainly  due  to  their  conflicting  design  requirements  (Caldwell  and  Tsagarakis  2002).  Widely 
accepted  problems  of  the  underactuated  DC  motor  solution  are  the  low  dexterity,  weight  and  noise,  as  well  as 
stiffness  and  size.  Thus,  finding  alternative  types  of  actuation  and  looking  for  a  higher  DOF  mechanism  becomes 
critical  for  enhancing  prosthesis  functionality  and  acceptance. 


Literature  Review  Conclusions 

Generally  the  problems  or  the  limitations  in  the  prosthetic  hand  design  era  can  be  seen  in  three  ways 

1. The  mechanical  structure  /  the  mechanism  to  achieve  all  the  DOF  that  the  human  hands  possess:  The 

problems  that  hinders  researchers  from  proposing  new  design  or  mechanism  are  due  to  limitation  of 
technology  such  as  getting  of  actuators  with  low  cost,  miniature  size,  high  power  output,  easy  to  control 
etc.  the  other  problem  also  might  be  due  to  the  high  cost  associated  with  the  manufacturing  of  fancy  shape 
and  intricate  objects. 

2.  The  actuation  system:  as  it  is  addressed  above  most  of  the  available  actuators  are  bulky  and  heavy  to  put 

them  in  the  palm  or  in  the  fingers.  Therefore  people  look  a  transmission  mechanism  to  locate  the  motors 
remotely  like  in  a  forearm.  While  doing  these  a  lot  of  issues  will  come  like  noise  if  we  use  gears,  screws, 
and  friction  when  we  use  belts  and/or  tendons,  hence  a  lot  of  power  loses  and  less  life  of  mechanical 
members  will  result. 

3.  Control  and  sensory  feed  back  problems. 

Due  to  the  above  and  other  reasons  most  of  the  currently  available  designs  are  limited  to  underactuated  type  so  that 
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to  minimize  control  complexity,  mainly  using  tendons;  what  makes  them  different  is  the  actuation  system  they 
employ,  the  sensors  and  the  control  scheme.  Even,  if  we  see  hands  developed  purely  from  kinematic  links  almost 
most  of  them  are  made  from  revolute  or  prismatic  joints  which  can  easily  be  actuated  with  conventional  actuators. 
While  trying  to  minimize  control  complexity  the  hands  are  also  loosing  dexterity. 

Hence,  based  on  supportive  literature  review  and  natural  hand  biomechanics  and  anatomical  data  we  are  working  on 
studying  and  specifying  prosthesis  architecture. 

We  have  structured  our  design  approach  in  two  ways, 

1.  Using  real-time  capturing  of  the  fingers  motion  by  camera  system  and  applying  image  processing  technique 

to  filter  out  the  required  data  points  of  the  joint  angels.  Having  these  data  as  an  input,  a  mechanism 
particularly  to  mimic  the  thumb  motion  will  be  developed.  The  study  of  the  human  hand  kinematics  can 
offer  interesting  solutions  for  the  human  hand  prosthesis  development  or  in  fields  like  computer  animation 
or  gesture  recognition  that  add  a  greatest  value  for  the  assistive  medical  device  and  upper  limb 
rehabilitation  engineering  techniques  as  well.  A  kinematic  synthesis  based  on  the  anthropomorphic  data, 
having  the  same  elements  and  approximated  motions  can  successfully  lead  to  develop  a  functional 
prototype  capable  to  copy  as  much  as  possible  the  natural  model. 

2.  Using  a  systematic  approach  based  on  modification  of  existing  devices  for  the  synthesis  of  possible  structures 

of  mechanisms  for  prosthetic  applications  (using  the  techniques  used  in  (Chang  and  Tseng  2004),  (Martell 
and  Gini  2007)  and  others). 

Prototype  2  Design:  Thumb  Design 

A  major  factor  in  the  evolutionary  change  in  hand  use  was  the  development  of  the  opposable  thumb,  which  resulted 
in  a  remarkable  increase  in  the  versatility  of  hand  function.  Opposition  involves  flexion,  abduction,  and  medial 
rotation  of  the  thumb  so  that  the  pulp  surface  can  make  contact  with  the  terminal  phalanges  of  one  or  all  of  the 
remaining  digits.  The  articulation  of  the  thumb  metacarpal  (the  proximal  bone  of  the  thumb)  and  the  trapezium  (a 
bone  of  the  wrist),  in  what  is  known  as  a  saddle  joint,  enables  the  thumb  to  rotate  45°  around  its  longitudinal  axis. 
This  movement  of  opposition  is  essential  for  effective  handling  and  exploration  of  small  objects  and  greatly 
facilitated  primate  tool  use  and  construction. 

The  kinematic  properties  of  the  thumb  should  mimic  those  of  a  human  thumb.  If  the  anatomical  joints  could  be 
considered  as  revolute  joints,  matching  the  robotic  thumb  joints  to  the  anatomic  joints  would  create  the  proper 
relative  motions  between  the  bones.  The  human  thumb  has  been  described  by  a  variety  of  kinematic  models 
(Kapandji,  1970),  (Landsmee  1989),  (Callahan  A.  D  1990)  and  (Cooney  1981).  Although  several  researchers 
describe  the  IP  joint  as  having  one  rotational  degree  of  freedom  (Kapandji,  1970),  (Landsmee  1989)  and  (Cooney 
1981),  there  is  less  consensus  on  the  characterization  of  the  MCP  and  CMC  joints.  Previous  research  investigating 
thumb  motion  considered  both  the  MCP  and  CMC  joints  to  be  universal  joints,  which  have  two  perpendicular  and 
intersecting  axes. 


MP:  Metacarpo¬ 
phalangeal 


IP;  Intel  phalangeal 


Figure  14:  Configuration  of  the  joint  variables  of  the  thumb  (Arimoto  2007)] 

(where  q0  denotes  the  angle  of  rotation  around  the  x-axis  and  qu  q2  and  q3  denote  the  angles  of  rotation  around  the  z- 
axis  perpendicular  to  the  xy-plane  ). 
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Most  of  the  currently  available  hands  have  a  thumb  with  four  DOFs  as  shown  in  Figure  14.  However,  the  human 
thumb  is  more  realistically  modeled  with  more  than  4DOF  ,and  for  this  reason  a  number  of  researchers  are  still 
working  on  it  to  accurately  replicate  its  DOFs  and  make  ease  of  grasping  and  manipulating  objects.  In  (J.  Veronica 
2004),  five-hinge  serial  linkage  model  structure  (Figure  15)  has  been  used  to  model  the  thumb. 


Figure  15:  Thumb  kinematics  modeled  as  a  serial  chain  of  five  hinges 

The  coupling  mechanism-based  prosthetic  hand  presented  in  (Yi-wei  2010)  uses  a  four-bar  linkage  mechanism  to 
design  each  finger  and  also  to  maximize  the  interaction  between  the  thumb  and  other  fingers,  a  spherical  joint  driven 
by  4-bar  linkages  has  been  used  for  the  thumb. 

Prototype  2  Design:  Advances  in  Kinematic  Synthesis  of  Tree-like  Kinematic  Chains  for  Exact  Trajectories 

Kinematic  synthesis  of  articulated  structures  is  a  promising  field  for  the  design  of  complex  systems.  However,  it  is 
still  at  its  first  stages  when  we  deal  with  spatial  motion  and  go  beyond  simple  serial  chains.  This  is  due  to  the  great 
difficulty  in  stating  meaningful  equations,  as  well  as  the  dimensionality  of  the  problem,  which  renders  most  solvers 
useless. 

Here  we  have  attacked  both  these  problems  in  order  to  develop  a  design  tool  to  be  used  for  anthropomorphic  tasks. 
On  one  hand,  we  are  developing  a  formulation  based  on  finite  screw  theory,  able  to  describe  the  geometry  of  the 
workspace  or  a  trajectory  within  the  workspace  of  any  spatial  chain.  This  procedure  is  used  to  perform  exact 
synthesis  and  to  simplify  the  formulation  of  the  equations.  On  the  other  hand,  a  general  kinematic  solver  has  been 
developed  that  can  deal  with  the  synthesis  of  tree-like  kinematic  structures,  such  as  that  of  the  human  hand.  It  is 
important  to  point  out  that  this  is  the  first  time  that  the  synthesis  of  tree -like  articulated  systems  is  attempted,  with 
successful  results  and  some  surprising  conclusions. 

Finite-screw  system  kinematics 

Given  a  trajectory  such  as  the  one  presented  in  Error!  Reference  source  not  found.Figure  16  for  a  trajectory  of  the 
thumb,  it  is  advantageous,  from  the  synthesis  point  of  view,  to  work  with  relative  displacements.  In  Figure  17  we 
express  the  displacements  of  the  thumb 4 s  proximal  phalanx  trajectory  as  screw  axes  with  a  pitch  (the  screw  lengths 
are  proportional  to  the  pitch).  We  can  see  that  the  screw  axes  of  the  displacements  with  their  pitches  generate  a 
screw  surface.  We  use  this  description  of  the  limb  motion  to  minimize  the  difference  with  respect  to  the  motion  of  a 
given  linkage,  the  articulated  system  chosen  to  be  used  in  the  prosthetic  hand. 
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Figure  16:  Thumb  trajectory 


Figure  17:  Screw  surface  corresponding  to  the  previous  thumb  trajectory 


Using  screw  systems  such  as  the  screw  surface  in  the  figure,  we  are  able  to  extend  the  known  theory  of  finite- 
position  synthesis  to  the  design  of  whole  trajectories  or  subspaces. 

The  approach  used  in  this  paper  for  stating  design  equations  is  based  on  generalizing  the  standard  inverse  kinematics 
of  robotics.  We  equate  the  forward  kinematics  of  a  serial  chain  to  a  set  of  goal  displacements  and  consider  the 
structural  parameters  also  as  variables.  So  far,  the  most  efficient  formulation  for  our  purposes  consists  of  stating  the 
forward  kinematics  of  relative  displacements  using  the  even  Clifford  subalgebra  of  the  projective  space  C+(P3),  also 
known  as  dual  quaternions  (Perez-Gracia  2011). 

For  a  serial  chain  with  n  joints,  in  which  each  joint  can  rotate  an  angle  Oi  and  slide  a  distance  di,  around  and  along 
the  axis  Si,  i  =  l,...,n,  the  forward  kinematics  of  relative  displacements  (with  respect  to  a  reference  position)  can  be 
expressed  as  the  composition  of  Clifford  algebra  elements  corresponding  to  the  motion  of  each  joint.  Let  00  and  dO 
be  the  joint  parameters  of  this  chain  when  in  the  reference  configuration.  Then,  the  movement  from  this  reference 
configuration  is  defined  by 


Q(  A0)  =  e 


Mjc  ^2c 

y-Si  e2S2  ., 


A0j 

e  2 


c  A0i  .  A0] 

b"  =  (cos  -2-  +  sm  — —  S 


w  A02  .  A02q  .  .  A0„  .  A0„ 
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where  the  hat  indicates  dual  joint  variables.  This  formulation  can  be  stated  for  any  serial  chain,  and  we  will  see  later 
that  it  can  be  extended  to  other  topologies. 

In  order  to  create  design  equations  for  the  kinematic  synthesis,  we  equate  these  forward  kinematic  equations  to  a  set 
of  positions  that  we  want  to  reach, 


7  =  2,  ...,m 


It  is  straightforward  now  to  count  for  the  number  of  equations  and  unknowns,  in  order  to  establish  the  maximum 
number  of  positions  we  can  define.  This  is  the  formulation  of  the  exact  synthesis  problem. 

A  fmte-screw  surface  is  a  ruled  surface  in  which  the  lines  with  their  associated  pitch  correspond  to  relative 
displacements.  A  screw  surface  will  be  a  screw  system  if  it  is  closed  under  addition  and  scalar  multiplication,  that  is, 
if  every  screw  of  the  set  can  be  written  as  a  linear  combination  of  screws  belonging  to  it.  The  finite  screw  systems 
corresponding  to  serial  chains  can  be  obtained  in  a  straightforward  way  from  the  dual  quaternion  formulation  of  the 
forward  kinematics;  in  fact,  they  are  subspaces  of  this  algebra.  Using  this  formulation,  a  formula  can  be  worked  out 
that  relates  the  magnitude  (in  this  case,  unique)  to  the  screw.  When  dealing  with  parallel  robots,  additional 
conditions  must  be  added  in  order  to  define  the  screw  surface. 

It  is  easy  now  to  generate  trajectories  and  to  transform  them  from  displacement  to  screw  in  order  to  create 
trajectories.  At  the  same  time,  given  the  fact  that  they  are  subspaces,  using  only  a  base  for  the  system,  we  can  apply 
a  simplified  finite -position  dimensional  synthesis  algorithm.  This  method  has  been  developed  in  (Perez-Gracia 
2011)  for  the  synthesis  of  a  spatial  overconstrained  linkage,  the  spatial  RPRP.  A  general  procedure  is  being 
developed  following  this  line. 

When  completed,  this  formulation  will  avoid  the  use  of  computationally-costly  minimization  procedure  in  order  to 
synthesize  articulated  fingers  to  follow  a  trajectory  of  poses.  It  will  be  applied  to  the  design  of  the  thumb  in  the  first 
place. 

General  kinematic  synthesis  solver  for  tree-like  articulated  system 

For  the  first  time  we  are  developing  a  tool  that  will  allow  the  kinematic  synthesis  of  tree -like  articulated  systems 
with  variable  number  and  type  of  joints.  This  is  the  outcome  of  the  job  being  developed  in  (Simo-Serra,  Moreno- 
Noguer  and  Perez-Gracia  2011  ).  The  kinematic  synthesis  problem  is  again  expressed  equating  the  forward 
kinematics  in  Clifford  algebra  form  to  a  set  of  finite  positions.  In  this  case,  we  are  developing  a  mixed  solver,  based 
on  genetic  algorithms  and  paired  with  a  local  Levenberg-Marquardt  solver.  Currently  this  is  being  applied  to  a  hand 
skeleton  which  is  anthropomorphic  in  its  degrees  of  freedom.  However,  the  huge  number  of  solutions  of  the  problem 
poses  an  additional  difficulty  if  we  want  to  select  something  for  prosthetic  applications. 

The  raw  application  of  the  solver  to  fingertip  trajectories  from  human  hands  leads  to  a  big  number  of  articulated 
structures,  which  are  highly  non-anthropomorphic.  Figure...  shows  one  of  these  results.  In  order  to  get 
anthropomorphic  solutions,  we  are  working  on  adding  additional  geometric  constraints  to  the  model.  One  important 
theoretical  result  out  of  this  research  is  the  proof  that,  despite  common  belief,  the  hand/wrist  complex  is  not  a 
redundant  kinematic  structure  when  the  tasks  of  more  than  one  fingertip  are  considered  simultaneously. 
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Figure  18:  Input  hand  model  (green)  and  one  non-anthropomorphic  solution  (grey) 

The  actual  model  of  the  hand  that  the  solver  is  working  with  has  26  degrees  of  freedom  and  handles  27  discrete 
poses  of  the  hand  for  finding  a  solution  set  of  dimension  zero  (that  is,  discrete  number  of  solutions).  This  implies  a 
system  with  more  than  800  variables  and  more  than  1000  equations,  all  highly  nonlinear.  The  solution  time  for  this 
model  is  about  60  hours  on  a  single  computer;  it  is  now  being  implemented  in  a  computer  cluster. 

This  solver  is  an  important  tool  to  test  different  chain  topologies  in  order  to  find  new  designs  for  some  of  the 
components  of  the  hand.  It  has  been  implemented  so  that  it  can  be  applied  to  the  design  of  any  articulated  system. 

Goal  3,  Objective  3:  Manipulation  of  Grasped  Objects 

The  main  reason  behind  studying  human  grasping  and  manipulation  characteristics  within  this  project  is  to 
complement  the  input  from  the  user  when  this  is  defective,  noisy  or  just  missing  due  to  the  degree  of  amputation. 
We  are  focusing  on  identifying  implicit  functions  and  workspace  limits  that  will  allow  us  to  decrease  the  degree  of 
the  configuration  space  of  the  hand  and  the  dimension  and  extension  of  its  Cartesian  manipulation  manifold.  In  the 
study  of  finger 4 s  kinematics,  it  is  obvious  to  consider  the  constraints  as  finger  motion  is  constrained  so  that  the  hand 
cannot  make  arbitrary  gestures.  There  are  many  examples  of  such  constraints.  For  instance,  fingers  cannot  bend 
backward  too  much  and  the  pinky  finger  cannot  be  bent  without  bending  the  ring  finger.  The  natural  movements  of 
human  hands  are  implicitly  defined  by  such  motion  constraints. 

In  order  to  do  so,  we  are  developing  an  experimental  setup  in  order  to  obtain  accurate  data  on  limit  angles  for  each 
degree  of  freedom,  and  in  order  to  identify  motion  primitives.  This  last  topics  follows  recent  research,  in  which  it  is 
hypothesized  that  the  hand  motion  is  created  on  the  hand  structure  and  can  be  divided  in  a  series  of  motion 
primitives,  which  could  be  identified  using  principal  component  analysis,  see  (M.  Gabiccini,  2010) 

The  main  problem  with  the  experimental  proof  and  finding  of  these  synergies,  or  motion  primitives,  is  the  lack  of  a 
good  system  for  hand  data  capture.  Most  of  the  existing  systems  either  restrict  the  hand  motion  or  miss  some  of  its 
degrees  of  freedom.  Due  to  this  reason,  we  have  focused  our  efforts  in  developing  a  non-contact,  accurate  motion 
capture  and  identification  system  for  the  hand. 

Computer  Vision  System  to  Determine  Kinematic  Information 

In  order  to  create  a  prosthetic  hand  that  mimics  the  human  hand  in  every  way,  almost  all  hand  motions  and  the 
related  EMG  signals  need  to  be  studied.  The  complex  hand  motion  requires  a  sensing  system  capable  of  following 
its  many  degrees  of  freedom  without  interfering  with  the  motion.  This  gives  rise  to  a  problem  that  can  be  solved 
using  image  processing  and  computer  vision  techniques. 
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The  development  of  this  system  follows  on  the  results  obtained  during  Phase  I  of  this  project,  and  builds  on  the 
knowledge  and  the  learning  from  the  errors  made  during  that  first  attempt. 

To  determine  the  articulation  of  fingers,  the  kinematical  structure  of  the  hand  is  modeled.  In  our  research,  the 
skeleton  of  a  hand  is  abstracted  as  a  stick  figure  with  each  finger  as  a  kinematical  chain  with  base  frame  at  the  palm 
and  each  fingertip  as  the  end-effector.  This  model  is  constrained  by  constraints  referred  to  the  limits  of  the  range  of 
finger  motions  as  a  result  of  hand  anatomy,  and  by  constraints  due  to  the  limits  imposed  on  joints  during  finger 
motion  (intra  and  inter  finger  constraints).  This  initial  skeleton  will  be  adjusted  to  the  real  anatomy  of  the  subject 
using  a  kinematic  synthesis  algorithm,  and  once  the  model  of  the  hand  is  accurate,  it  is  straightforward  to  calculate 
the  joint  angles  for  each  position  of  the  fingertips.  The  non-contact  data  will  be  obtained  using  a  camera  system. 

Computer  Vision  System:  Experimental  Procedures  and  Analysis 

The  pose  of  a  calibrated  camera  can  be  uniquely  determined  from  a  minimum  of  four  coplanar  but  non-collinear 
points.  In  practice,  there  are  many  applications  of  camera  pose  tracking  from  planar  targets  and  there  are  also  a 
number  of  recent  pose  estimation  algorithms,  in  our  research,  in  order  to  compute  pose  for  3D  to  2D 
correspondences  of  a  planar  target,  the  algorithm  used  is  based  on  Schweighofer  (Schweighofer  2006) 

The  whole  experimental  procedure  is  explained  as  follow,  two  squares  with  corners  marked  are  attached  to  the 
thumb  and  to  the  base  of  the  thumb/wrist/as  shown  in  Figure  19  .  Then  the  thumb  moved  to  its  different 
positions/reaches.  While  moving  the  thumb  different  frames  are  captured  using  the  camera  interfaced  with  computer 
and  using  Lab  VIEW  block  diagrams  (Figure  20).  This  Lab  VIEW  program  captures  and  saves  multiple  images  in  to 
a  data  storage  folder  on  the  hard  disk.  Once  the  frames  are  saved,  sample  frames  from  different  orientation  of  the 
thumb  have  been  selected  for  processing. 

The  following  set  of  data  is  provided  to  the  algorithm  in  Matlab 

1.  2D  coordinates  in  pixels  and  3D  coordinates  of  the  four  points  (corners  of  the  square). 

2.  Intrinsic  parameters  of  the  camera 


Figure  19:  Square  Placement  for  Studying  Grasping  Motion 


Figure  20:  LabVIEW  Block  Diagram  for  Continuous  Image  Capture 
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For  each  squares  attached  to  the  hand  we  get  the  rotation  and  translation  matrixes  of  the  point  with  respect  to  the 
camera  reference  frame.  Having  this  information  the  orientation  of  the  tip  of  the  thumb  with  respect  to  the  wrist  has 
been  tabulated. 

So  far  the  system  is  working  for  a  single  marker  on  a  single  image.  Current  work  is  being  developed  in  adding 
cameras  to  the  system  for  greater  resolution,  and  on  the  marker  tracking  algorithms.  The  results  of  this  hand  motion 
identification  system  will  be  used  for  identification  of  hand  primitives  and  joint  ranges,  as  input  data  for  the 
synthesis  algorithm  in  the  design  of  Prototype  2,  and  also  to  study  the  correspondence  between  the  EMG  signals  and 
the  motion  of  each  individual  finger  and  degree  of  freedom. 
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Goal  4:  Biocompatibility  and  Signaling 

Overview:  A  major  unresolved  biocompatibility  issue  pertains  to  how  tissue  and  cellular  inflammatory  responses 
are  elicited  by  artificial  implant  material(s).  We  are  developing  a  — Sesor  on  Cells  on  Skin  Model”  to  facilitate  the 
investigation  of  this  biocompatibility  issue.  We  have  made  significant  progress  in  cell  model  development.  We  have 
developed  new  cell  models  of  peripheral  nervous  system  consisting  of  dorsal  root  ganglion  neurons  and  Schwann 
cells.  The  new  models  are  suitable  for  elucidating  nerve  degeneration  and  regeneration.  We  are  characterizing  these 
new  models  further.  We  have  developed  several  -Hsiart”  biosensors  for  tissue  engineering  and  biocompatibility 
studies  and  related  biomedical  engineering  applications.  In  parallel  with  cell  model  and  sensor  development,  we 
have  investigated  the  molecular  mechanisms  underlying  the  cytotoxicity  of  silicon  dioxide  nanoparticles  in  neural 
cells.  Our  extensive  literature  survey  indicates  that  our  studies  are  truly  novel  and  will  lead  to  further  ground¬ 
breaking  discoveries  because  the  whole  important  issue  of  how  tissue  and  cellular  inflammatory  responses  are 
elicited  by  implant  material(s)  have  not  been  resolved  and  is  poorly  understood. 

Current  artificial  organ  and  tissue  engineering  research  (e.g.,  artificial  bone  and  joint  implants)  reveals  that  over  time 
the  artificial  implants  fail  to  function  even  though  over  a  short  initial  period  after  they  were  implanted,  such  devices 
appear  to  function  as  the  biomedical  engineers,  who  had  designed  them,  expected.  The  findings  of  numerous  studies 
indicate  that  the  failure  of  the  implants  (despite  their  initial  success)  were  attributed  to  the  fact  that,  over  time,  the 
cells/tissues  immediately  surrounding  and/or  in  contact  with  the  implants  appear  to  develop  inflammatory 
reactions”  to  the  implants  thereby  compromising  the  implants4  performance  (see  Lai  et  al.,  2007a  &  ref.  therein). 
Furthermore,  if  the  implants  act  as  sensors,  the  inflammatory  reactions”  exhibited  by  the  cells/tissues  surrounding 
the  implanted  sensors  induce  interference  of  the  signal  transmission  of  the  implants.  However,  inflammatory 
reactions”  exhibited  by  the  cells/tissues  surrounding  the  implanted  sensors  are  poorly  understood  and  remain  to  be 
major  challenges  for  tissue  engineers  and  other  biomedical  engineers  who  design  such  implant  devices.  Moreover, 
our  extensive  searching  in  the  literature  reveals  that  there  is  a  paucity  of  literature  on  robotic  implants  and  tissue 
inflammatory  responses  (Lai  et  al.,  2007a).  Nevertheless,  nanoparticles  and  other  nanomaterials  have  been 
increasing  employed  to  manufacture  implants  and  biosensors.  There  is  emerging  concerns  that  some  nanomaterials 
may  post  health  hazards  to  humans  because  of  recent  findings  of  nano  toxicology  (Lai  et  al.,  2007a,b).  Consequently, 
in  order  to  circumvent  such  biocompatibility  issues  in  the  design  of  robotic  implants,  it  is  imperative  to  elucidate  the 
mechanisms  by  which  such  — iflammatory  reactions”  exhibited  by  the  cells/tissues  surrounding  the  implants  arise 
and  elucidate  the  putative  nanotoxicity  of  nanomaterials  employed  to  fabricate  implants  and  biosensors.  Thus,  our  in 
vitro  model  (see  Fig.  1  below)  allows  us  to  address  these  physiologically  and  pathophysiologically  important 
mechanistic  issues.  For  phase  II  of  our  projects,  the  goals  are  to:  (1)  Continue  to  develop  the  -sensor  on  cells  on  skin 
model,”  especially  stages  two  and  three  of  the  model  development  (see  Addenda  of  Full  Proposal);  (2)  Further 
characterize  the  putative  nanotoxicity  of  nanoparticles  and  other  nanomaterials  commonly  employed  to  fabricate 
biosensors  and  implants;  (3)  Detect  signal  mechanisms  that  influence  cell  growth  and  suppression  such  as  apoptosis; 
(4)  Couple  enzymes  on  sensors  for  signal  identifications  and  amplifications;  and  (5)  Detect  signals  for  biological 
growth  and  interferences  with  quantifiable  parameters  such  as  current,  voltage,  pH,  etc.  between  various  cell  types 
of  different  organs  of  origin. 

The  following  are  the  deliverables: 

1)  Continue  to  develop  the  — seaor  on  cells  on  skin  model,”  especially  stages  two 
and  three  of  the  model  development  (see  Addenda  of  Full  Proposal); 

2)  Further  characterize  the  putative  nanotoxicity  of  nanoparticles  and  other 
nanomaterials  commonly  employed  to  fabricate  biosensors  and  implants; 

3)  Detect  signal  mechanisms  that  influence  cell  growth  and  suppression  such  as  apoptosis; 

4)  Couple  enzymes  on  sensors  for  signal  identifications  and  amplifications;  and 

5)  Detect  signals  for  biological  growth  and  interferences  with  quantifiable  parameters  such  as  current,  voltage, 
pH,  etc.  between  various  cell  types  of  different  organs  of  origin. 

We  have  continued  to  make  progress  in  studies  outlined  in  Goal  4  of  our  proposal  and  have  summarized  the  results 
obtained  during  this  period  in  the  following  sections.  We  have  been  productive  despite  the  gap  in  funding  between 
the  ending  of  Phase  I  and  initiation  of  funding  of  Phase  II:  we  have  presented  seven  papers  at  national  and 
international  meetings  and  have  seven  journal  articles,  one  review,  &  seven  conference  proceedings  papers 
published  or  in  press  (see  Publications  below). 
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Set  up  of  the  “Sensor  on  Cells  on  Skin  Model”: 

Based  on  our  extensive  experience  in  working  with  mammalian  (including  human)  cells  in  culture  (Dukhande  et  ah, 
2006;  Hertz  et  al.,  1998;  Isaac  et  ah,  2006;  Isaac  et  ah,  2007;  Lai  et  al.,  2008,  2010;  Malthankar  et  al.,  2004;  Puli  et 
al.,  2006),  we  envisaged  that  we  will  undergo  three  stages  to  develop  our  -sensor  on  cells  on  skin  model:”  (i)  stage 
one  involves  the  development  of  different  human  and  other  mammalian  cells  (including  nerve,  muscle,  and  skin 
cells)  in  co-culture;  (ii)  stage  two  requires  the  juxtaposition  of  cells  in  relation  to  the  membrane  as  illustrated  in 
Figure  1.  below  (this  figure  was  included  in  our  proposal);  and  (iii)  stage  three  involves  designing  the  sensor(s)  and 
fitting  them  into  the  — seaor  on  cells  on  skin  model”  (as  shown  in  Fig.  1). 

We  successfully  competed  (i)  stage  one  of  setting  up  the  model  as  depicted  above.  More  recently,  we  have 
begun  developing  new  cell  models  employing  dorsal  root  ganglion  neurons  and  Schwann  cells,  both  are  neural  cells 
from  the  peripheral  nervous  system  (see  Appendices  2  &  6).  Both  dorsal  root  ganglion  neurons  and  Schwann  cells 
are  important  and  relevant  cell  types  for  investigating  biocompatibility  of  biomaterials.  We  have  therefore  started 
characterizing  these  cell  types. 


Fig.  1  -Sensor  on  Cells  on  Skin  Model.”  The  sensor  (or  implant:  blue  rectangle)  is  in  contract  with  a  monolayer 
of  muscle  or  nerve  cells  (gray  ovals),  which  are  in  contact  with  the  artificial  skin  (an  artificial  membrane  with 
specified  pores:  brown  rectangle),  (a)  stimulating  electrode;  (b)  sensor  signal  pick-up;  (c)  alternative  position  of 
sensor. 

As  shown  in  Figure  2,  the  population  of  dorsal  root  ganglion  neurons  showed  exponential  growth  in  vitro  as  a 
function  of  initial  seeding  density.  Similarly,  as  shown  in  Figure  3,  the  SI 6  Schwann  cells  also  showed 
exponential  growth  in  vitro  as  a  function  of  initial  seeding  density.  Once  we  have  established  their  growth 
patterns  in  culture,  we  are  now  in  a  position  to  further  characterize  them  in  terms  of  other  physiological  and  cell 
biological  functions. 

We  have  continued  the  studies  to  address  (ii)  stage  two  of  setting  up  the  model  and  (iii)  stage  three  of  designing 
and  characterizing  the  biosensors.  We  have  summarized  below  our  progress  in  developing  and  characterizing 
the  biosensors. 
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Growth  of  Dorsal  Root  Ganglion  Neurons 


Figure  2.  Growth  of  Dorsal  Root  Ganglion  Neurons  in  Culture. 


SI 6  Schwann  Cell  Calibration  Curve 


Figure  3.  Growth  of  SI 6  Schwann  Cells  in  Culture. 

Sensor  Development: 
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During  Phase  I  of  this  project,  we  studied  the  development  of  biosensor  platform  that  are  capable  of 
measuring  biometabolites  and  environmental  sensitive  species,  such  as  peroxide  and  nitrate/nitrate,  to  concentration 
in  the  order  of  ppb  (parts  per  billion)  or  lower. 

In  our  more  recent  development,  we  modified  our  platform  with  dendrimers  to  enhance  its  performance. 
Zero  and  second  generation  of  dendrimers  were  coated  on  the  surface  of  a  carbon  glassy  platform  electrode  modified 
with  GDH  (glutamate  dehydrogenase)  and  it  was  used  to  measured  ammonium,  a  common  biometabolite,  at  near 
neutral  pH  that  is  common  for  normal  bioactivities.  The  resulting  electrode  was  tested  with  ammonium 
concentrations  ranged  from  0.002  to  0.3  juM  with  satisfactory  results.  Figure  4  shows  the  cyclic  voltammograms  of 
various  ammonium  concentrations  measured  by  the  modified  glassy  carbon  electrode,  the  lowest  concentration  was 
2  nM.  As  indicated  in  the  voltammogram  in  the  reductive  curves  (upper  lines),  there  was  a  big  gap  (current 
difference)  between  the  blank  (of  buffer  solution)  and  the  first  reductive  curve  (2  nM).  Therefore,  it  is  possible  that 
the  modified  electrode  can  measure  ammonium  concentrations  down  to  subnanomolar  levels. 


\flrv 4M 1 


Figure  4.  Cyclic  voltammograms  (i/v)  of  gold  colloid-cysteamine/PAMAM_2  -modified  glassy  carbon 
electrode  measured  from-0.2  to  0.4  V  for  the  measurement  of  NH4+,  the  lowest  concentration  was  2  nM. 
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Figure  5.  Performance  differences  of  the  glassy  carbon  electrode  modified  by  different  materials:  1.  Modified  with 
cysteamine-Au-GDH/NADH;  2.  Cysteamine  /PAMAM_0-Au-GDH/NADH;  3.  PAMAM_0-Au-GDH/NADH;  4. 
Cysteamine  /PAMAM_2nd-Au-GDH/NADH;  5.  PAMAM_2nd  -Au-GDH/NADH. 

Figure  5  shows  that  there  were  distinctive  current  differences  for  the  reaction  of  ammonium  and  a-ketoglutarate 
when  the  electrode  was  modified  by  different  materials.  From  the  stand  point  of  detection,  within  the  five 
modifications,  sensor  coated  with  second  generation  of  PAM  AM  would  have  the  highest  sensitivity  (current  vs. 
concentration).  The  results  were  conceivably  due  to  the  higher  branching  property  of  PAMAM_2nd. 

As  shown  in  Figure  6,  PAM  AMs  can  be  attached  to  cysteamine  to  enhance  the  available  sites  for  the  electrons 
transfer  between  reactions,  in  this  case,  ammonium  and  a-ketoglutarate,  therefore  increase  the  detection  lower  limit. 


Figure  6.  Cysteamine -PAMAM-Au-GDH  coated  on  plastic  UV-VIS  cell  with  permanent  absorption  peak  at  278  nm. 
Various  lines  indicated  concentration  differences. 
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Figure  7.  Cyclic  voltammograms  (i/v)  of  gold  colloid-cysteamine/PAMAM_2ND-modified  glassy  carbon  electrode 
measured  for  the  measurement  of  a-ketoglutarate,  which  is  an  important  tricarboxylic  acid  cycle  intermediate  and  a 
common  metabolite. 


Figure  8.  Cyclic  voltammograms  of  GDH/NADH  modified  glassy  carbon  electrode  (at  low  and  high  voltage  range) 
for  current  vs.  real  concentration  of  a-ketoglutarate  at  low  and  high  voltage  for  GDH  and  NADH  modified 
electrodes.  1.  Glassy  carbon  electrodes  modified  with  cysteamine  -Au-GDH.  2.  Glassy  carbon  electrodes  modified 
with  cysteamine  /PAMAMO  -Au-GDH.  3.  Glassy  carbon  electrodes  modified  with  cysteamine  /PAMAM_2nd  -Au- 
GDH.  4.  Glassy  carbon  electrodes  modified  with  PAMAM_2nd  -Au-GDH.  5.  Glassy  carbon  electrodes  modified 
with  AMAMO  -Au-GDH. 

Figures  7  and  8  show  that  there  were  distinctive  current  differences  for  the  reaction  of  ammonium  and  a- 
ketoglutarate  when  the  electrode  was  modified  by  different  materials.  Because  a-ketoglutarate  is  an  important 
tricarboxylic  acid  cycle  intermediate  and  a  common  metabolite,  the  modifications  as  indicated  in  the  two  figures 
suggested  that  the  biosensor  has  good  capacity  for  detecting  a  range  of  concentrations  of  this  metabolite.  However, 
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as  shown  in  Figure  5,  the  current-voltage  relationship  is  nearly  linear  after  the  initial  extreme  low  concentrations. 
Thus,  one  should  be  cautious  in  selecting  for  the  i/v  ratio  depending  on  the  applications  (for  detection  of  extreme 
low  and  higher  concentrations). 

In  our  more  recent  development,  we  modified  our  platform  to  include  the  parallel  development  of  biosensors  and 
fuel  cells.  As  in  most  biosensors,  the  detection  principles  are  based  on  redox  reactions  of  enzymatic  reactions,  which 
is  the  same  principle  used  in  most  biofuel  cell  and  how  biological  systems  obtain  energy  to  sustain  live  functions. 
For  the  sensors  we  previously  developed  that  utilize  the  enzymes,  LHD  and  GDH,  for  detections  of  various 
metabolites,  biochemical  reactions  that  involved  these  two  enzymes  can  also  be  converted  to  a  biofuel  cell  system: 

For  the  biofuel  cell  system,  the  anodic  half-cell  was  made  by  an  Au  electrode  coated  with  lactate  dehydrogenase 
(LDH)  submerged  in  solution  containing  lactate  and  nicotinamide  adenine  dinucleotide  (NAD+),  thus,  for  the  anodic 
half  reaction: 

LDH 

Anode:  Lactate  +  NAD+  — *  pyruvate  +  NADH  +  2H+  +  2e"  (1) 

As  lactate  was  oxidized  into  pyruvate  in  the  anode,  the  oxidative  potential,  E0‘(25  °C),  would  be  0.19  V. 

The  cathodic  half-cell  was  made  by  a  glassy  carbon  electrode  (GCE)  coated  with  glutamate  dehydrogenase  (GDH) 
submerged  in  solution  containing  ammonia  and  reduced  nicotinamide  adenine  dinucleotide  (NADH),  the  cathodic 
half  reaction  was: 

CrDH 

Cathode:  a-Ketoglutarate  +  NH4+  +  NADH  +  H+  +  2e"  — *  glutamate  +  NAD+  +  H20  (2) 

As  a-ketoglutarate  was  reduced  into  glutamate  in  the  cathode,  it  generated  a  reductive  potential,  E04(25  °C),  of  -0.14 
V. 


Combining  Equation  (1)  and  (2),  the  total  potential  for  the  fuel  cell  with  the  anode  and  cathode  above  was  0.05  V 
(per  mole).  In  general,  biological  reactions  in  nature  do  not  release  large  amount  of  energy  for  each  reaction  step  to 
preserve  energy  utilization  efficiency.  The  overall  reaction  for  the  biofuel  cell  in  our  system  was: 

LDH  GDH 

Lactate  +  a-ketoglutarate  +  NH4+ - *  pyruvate  +  glutamate  +  H+  +  H20  (3) 

Since  the  A  E04  of  Equation  (3)  was  greater  than  0,  the  reaction  was  spontaneous  and  would  proceed  as  written  from 
left  to  right. 

Anodic  half-cell 

Figure  9  shows  the  oxidative  responses  of  the  modified  Au  electrode  with  nanogold  particles,  cysteamine  and 
LDH.  As  seen,  the  characteristic  peak  of  lactate  conversion  to  pyruvate  was  detected  at  250  mV  and  the 
accumulative  current  increased  linearly  with  added  lactate  instantaneously  [6].  It  verified  that  the  modified  anodic 
electrode  was  functioning  as  expected  in  Equation  (1). 

Cathodic  half-cell 

Figure  10  is  the  reductive  response  of  a  GCE  modified  with  2nd  generation  of  PAMAMs,  nanogold  particles,  and 
GDH.  The  cathodic  half-cell  reaction  functioned  more  efficiently  at  lower  concentrations  of  a-ketoglutarate  and 
became  less  effective  as  the  accumulative  concentration  of  a-ketoglutarate  was  higher  (>0.025  mM). 
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Figure  9.  Voltammetric  responses  of  an  Au  electrode  coated  with  cysteamine,  Au  nanoparticles,  and  LDH  at  pH  8. 
Responses  were  stepwise  additions  of  lactate  at  l.Ox"4  mol/ml  with  linear  current  increments  (oxidative  peaks)  at  250 
mV. 
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Figure  10.  Reductive  responses  of  a  glassy  carbon  electrode  modified  with  PAM  AM,  Au  nanoparticle  and  GDH 
measured  at  0.7  V  in  solution  of  ammonia  and  a-ketoglutarate. 

Our  biofuel  cell  system  would  generate  a  relative  low  potential  (0.05V),  but  the  reactants  are  all  readily  available 
from  our  body  which  make  it  closer  to  reality  for  the  system  to  be  implanted  into  our  body  as  power  source  for 
biomedical  devices.  A  similar  approach  can  be  used  to  explore  enzymatic  reactions  within  human  body  that  can 
produce  higher  energy  output.  This  biofuel  cell  can  be  used  potentially  to  power  the  prosthetic  hand  this  grant  is 
developing. 

The  biosensor  platform  thus  far  was  proven  to  be  versatile  and  can  be  used  in  many  biomedical  and  environmental 
applications.  We  continue  to  work  on  the  development  of  these  smart  sensors:  we  are  particularly  focusing  on  the 
development  of  bionanosensors  for  tissue  engineering  and  biocompatibility  studies  and  related  biomedical 
engineering  applications  (see  Appendices  9  &13). 
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A.  Investigate  the  biocompatibility  of  materials  that  are  commonly  used  to  fabricate  sensors: 

Introductory  remarks.  One  established  approach  to  investigate  the  biocompatibility  of  materials  is  to 
determine  whether  or  not  such  materials  exert  cytotoxic  effects  on  mammalian  cells  (Lai  et  al.,  2008,  2010).  Two 
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nanomaterials  that  are  commonly  used  to  fabricate  nanobiosensors  are  silicon  dioxide  nanoparticles  and  titanium 
oxide  nanoparticles  (Lai  et  al.,  2008,  2010). 

Molecular  and  Signaling  Mechanisms  underlying  the  Cytotoxic  Effects  of  Metallic  Oxide  Nanoparticles. 

Nanomaterials  have  been  increasingly  used  in  industrial  applications  (e.g.,  drug  delivery,  additives  to  drugs  and 
cosmetics).  Because  of  their  wide  use,  occupational  exposure  to  nanomaterials  and  nanoparticles  may  pose  as  health 
risks.  Recent  studies  have  suggested  that  exposure  to  nanoparticles  may  induce  cytotoxic  effects  in  some 
mammalian  cell  types  (Lai  et  al.,  2010;  Puli  et  al.,  2006)  although  these  effects  have  not  been  systematically 
investigated. 

In  these  studies,  we  have  begun  to  systematically  investigate  the  molecular  and  signaling  mechanisms 
underlying  the  cytotoxic  effects  of  nanomaterials,  especially  nanoparticles,  on  human  neural  and  non-neural  cell 
types.  Here  we  report  on  some  of  the  mechanisms  responsible  for  the  cytotoxic  effects  of  silicon  dioxide  (Si02) 
nanparticles  on  human  astrocytoma  U87  cells  that  are  astrocytes -like. 
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Figure  11.  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  specific  activities  of  citrate  synthase  in  human 
astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon  dioxide  nanoparticles  for  48 
hours.  Then  the  activities  of  citrate  synthase  in  the  homogenates  of  treated  and  untreated  (i.e.,  control,  Ctrl)  U87  cells 
were  determined  as  described  in  Materials  and  Methods;  the  activities  of  citrate  synthase  were  expressed  per  mg  of 
homogenate  protein  as  specific  activities.  The  specific  activities  values  were  the  mean  ±  SEM  of  at  least  three 
separate  experiments;  Ctrl  represented  the  value  in  untreated  U87  cell  homogenate;  *p  <  0.05  versus  that  of  control 
cells. 

Because  cell  survival  critically  depends  on  mitochondrial  functions  maintained  at  a  normal  physiological  level,  we 
determined  the  effect  of  Si02  nanoparticles  on  mitochondrial  function  in  U87  cells  by  monitoring  the  activities  of 
one  key  enzymes  of  the  tricarboxylic  acid  (TCA)  cycle  namely,  citrate  synthase  (CS).  As  shown  in  Figure  11,  at 
treatment  concentrations  of  25-100  pg/mL  for  48  hours,  Si02  nanoparticles  induced  dose-related  increases  in  CS 
activities  in  U87  cells. 


Page  154  of  531 


Si02  nano  (fig/ml) 


£RK 


p-ERK 


p-actin 


Figure  12.  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of  ERK  and  phosphor-ERK  in  human 
astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon  dioxide  nanoparticles  for  48 
hours.  Then  cell  lysates  of  treated  and  untreated  (i.e.,  control)  U87  cells  were  prepared  as  described  in  Materials  and 
Methods.  The  expression  of  ERK  and  phosphor-ERK  (p-ERK)  was  determined  by  Western  blot  analysis  using  p- 
actin  as  the  loading  control:  Lane  marked  C,  lysate  of  untreated  or  control  U87  cells;  lane  marked  1,  lysate  of  U87 
cells  treated  with  silicon  dioxide  nanoparticles  at  1  pg/ml;  lane  marked  10,  lysate  of  U87  cells  treated  with  silicon 
dioxide  nanoparticles  at  10  jag/ml;  lane  marked  50,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at 
50  ju,g/ml.  The  blots  were  from  a  typical  experiment.  Two  other  experiments  yielded  essential  the  same  trend  of 
results. 


Treatment  of  U87  cells  with  Si02  nanoparticles  could  also  lower  the  survival  of  U87  cells  through  altering 
cell  signaling  pathway(s)  that  regulate(s)  cell  survival  and  proliferation  (7).  We  therefore  investigated  this  possibility 
by  examining  the  effect  of  these  nanoparticles  on  expression  of  ERK  and  phospho-ERK  proteins.  We  observed  that 
when  U87  cells  were  treated  with  1-50  pg/mL  of  Si02  nanoparticles  for  48  hours,  their  protein  expression  of 
phospho-ERK  showed  a  dose-related  decrease  while  that  of  ERK  protein  remained  essentially  unchanged  (Figure 
12),  suggesting  that  a  lowering  of  this  cell  survival/proliferation  signaling  mechanism  could,  at  least  in  part,  account 
for  the  dose-related  decrease  in  U87  cell  survival  induced  by  these  nanoparticles. 

Taken  together,  the  results  of  our  ongoing  studies  on  the  molecular  and  signaling  mechanisms  underlying 
the  cytotoxic  effects  of  silicon  dioxide  (Si02)  nanparticles  indicate  that  alterations  of  mitochondrial  bioenergetics 
and  cell  survival  signaling  underly  the  cytotoxic  effects  of  these  nanoparticles.  Moreover,  they  suggest  there  may  be 
other  such  mechanisms  present.  We  are  therefore  continuing  to  pursue  this  interesting  line  of  investigation. 
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Malate  Dehyrdogenase 


Figure  13.  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  specific  activities  of  citrate  synthase  in  human 
astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon  dioxide  nanoparticles  for  48 
hours.  Then  the  activities  of  malate  dehydrogenase  in  the  homogenates  of  treated  and  untreated  (i.e.,  control,  Ctrl) 
U87  cells  were  determined  as  described  in  Materials  and  Methods;  the  activities  of  malate  dehydrogenase  were 
expressed  per  mg  of  homogenate  protein  as  specific  activities.  The  specific  activities  values  were  the  mean  ±  SEM 
of  at  least  three  separate  experiments;  Ctrl  represented  the  value  in  untreated  U87  cell  homogenate;  *p  <  0.05  versus 
that  of  control  cells. 


Because  cell  survival  critically  depends  on  mitochondrial  functions  maintained  at  a  normal  physiological 
level,  we  determined  the  effect  of  Si02  nanoparticles  on  mitochondrial  function  in  U87  cells  by  monitoring  the 
activities  of  one  key  enzymes  of  the  TCA  cycle  namely,  malate  dehydrogenase  (MDH).  As  shown  in  Figure  13,  at 
treatment  concentrations  of  25-100  pg/mL  for  48  hours,  Si02  nanoparticles  induced  dose-related  increases  in  MDH 
activities  in  U87  cells. 

12  3  4 


Cytochrome  Oxidase 


(3-actin 


Figure  14.  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of  cytochrome  oxidase  subunit  II  in 
human  astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon  dioxide  nanoparticles  for 
48  hours.  Then  cell  lysates  of  treated  and  untreated  (i.e.,  control)  U87  cells  were  prepared  as  described  in  Materials 
and  Methods.  The  expression  of  cytochrome  oxidase  subunit  II,  encoded  by  mitochondrial  DNA,  was  determined  by 
Western  blot  analysis  using  P-actin  as  the  loading  control:  Lane  1,  lysate  of  untreated  or  control  U87  cells;  lane  2, 
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lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at  25  pg/ml;  lane  3  lysate  of  U87  cells  treated  with 
silicon  dioxide  nanoparticles  at  50  pg/ml;  lane  4,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at  100 
pg/ml.  The  blots  were  from  a  typical  experiment.  Two  other  experiments  yielded  essential  the  same  trend  of  results. 
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Figure  15.  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of  NADH  dehydrogenase  subunit  6 
(ND-6)  in  human  astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon  dioxide 
nanoparticles  for  48  hours.  Then  cell  lysates  of  treated  and  untreated  (i.e.,  control)  U87  cells  were  prepared  as 
described  in  Materials  and  Methods.  The  expression  of  NADH  dehydrogenase  subunit  6  (ND-6),  encoded  by 
mitochondrial  DNA,  was  determined  by  Western  blot  analysis  using  p-actin  as  the  loading  control:  Lane  1,  lysate  of 
untreated  or  control  U87  cells;  lane  2,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at  25  pg/ml;  lane 
3  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at  50  pg/ml;  lane  4,  lysate  of  U87  cells  treated  with 
silicon  dioxide  nanoparticles  at  100  pg/ml.  The  blots  were  from  a  typical  experiment.  Two  other  experiments 
yielded  essential  the  same  trend  of  results. 

On  the  other  hand,  at  treatment  concentrations  of  25-100  pg/mL  for  48  hours,  Si02  nanoparticles  induced 
dose-related  decreases  in  the  expression  of  cytochrome  C  oxidase  subunit  II  (a  mitochondrial  DNA-encoded 
peptide)  in  U87  cells  (Figure  14).  These  results  strongly  suggested  treatment  with  Si02  nanoparticles  altered  the 
normal  functioning  of  the  mitochondrial  genome  in  U87  cells. 

At  treatment  concentrations  of  25-100  pg/mL  for  48  hours,  Si02  nanoparticles  induced  dose-related 
decreases  in  the  expression  of  NADH  dehydrogenase  subunit  6  (ND  6)  (a  mitochondrial  DNA-encoded  peptide)  in 
U87  cells  (Figure  15).  These  results  strongly  suggested  treatment  with  Si02  nanoparticles  altered  the  normal 
functioning  of  the  mitochondrial  genome  in  U87  cells.  A  likely  consequence  of  this  effect  of  the  nanoparticles  is  the 
impairment  in  the  electron  transport  chain  of  the  mitochondria  in  U87  cells:  indeed,  the  nanoparticle-induced 
disruption  of  mitochondrial  respiratory  chain  structure  and  function  may  be  one  cause  of  energy  failure  that 
ultimately  led  to  the  death  of  U87  cells. 

Taken  together,  the  results  of  our  ongoing  studies  on  the  molecular  mechanisms  underlying  the  cytotoxic 
effects  of  silicon  dioxide  (Si02)  nanparticles  indicate  that  alterations  of  mitochondrial  bioenergetics  underly  the 
cytotoxic  effects  of  these  nanoparticles.  Moreover,  they  suggest  there  may  be  other  such  mechanisms  present. 
Consequently,  we  are  continuing  to  pursue  this  interesting  line  of  investigation. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


Goal  1:  EMG  Signal  Identification 

The  objectives  that  were  accomplished  for  the  Phase  II  of  the  -Smart  Prosthetic  Hand  Project”  for  the  EMG 
group  are  as  follows; 

1)  We  proposed  the  expansion  and  development  of  dynamical  models  for  relation  between  sEMG-force  and 
sEMG  and  finger  joint  angle.  We  have  successfully  developed  and  published  various  articles  on  the 
different  modeling  methods  and  sEMG  signal  processing  techniques  that  we  investigated  during  this  phase 
for  various  conferences  (Please  refer  publications: ) 

2)  Classification  based  on  degree  of  amputation  (physical  classification)  was  completed  using  three  female 
and  four  male  cadaver  specimens  (16  arms)  were  procured  from  the  existing  cadaver  holdings  at  Idaho 
State  University.  The  measurements  of  various  muscle  bellies  and  lengths  of  forearm  and  upper  arm  flexor 
grouping  made  are  reported  in  detail  in  section  2.1  of  this  draft. 

3)  A  very  thorough  investigation  of  the  different  types  of  spatial  filters  was  conducted  during  this  phase.  We 
investigated  1 1  types  of  spatial  filters  and  4  ISEK  standard  filters  for  sEMG  signal  processing.  A  detailed 
description  of  this  research  is  provided  under  section  2.2  of  this  draft.  We  improved  the  capabilities  of  the 
existing  spatial  filters,  available  in  the  literature  and  optimized  various  filter  masks,  to  obtain  significantly 
higher  values  of  model  fits  for  the  sEMG-force  relation.  In  all  under  publication  P 1  and  P2  we  tested  more 
than  10,000  different  Hammer  stein- Wiener  models. 

4)  Development  of  an  array  sensor  was  also  investigated  under  this  phase.  A  detailed  description  of  the 
research  conducted  is  provided  under  section  3  of  this  draft. 

5)  A  further  extension  to  the  development  of  the  array  sensor  was  developing  our  sensor  fusion  algorithm. 
This  was  also  successfully  completed  and  the  research  published  under  P3  and  P4.  A  detailed  report  on  this 
research  is  provided  under  section  3.2  of  this  draft. 

6)  Also,  modeling  of  the  fatigue  in  the  modeling  of  sEMG-force  was  accomplished  under  this  phase  for  an 
array  based  sensor  arrangement.  A  detailed  description  of  this  research  is  provided  under  section  4  and  5  of 
this  draft.  Please  also  refer  to  publications 

7)  Along  with  the  aforementioned  research  we  also  investigated  the  modeling  of  sEMG-joint  angle  of  the 
index  finger.  A  detailed  report  of  this  research  is  presented  under  section  6  of  this  draft.  Also,  the  results  of 
this  investigation  was  published  in  shown  under  reportable  outcomes. 
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Goal  2A:  Intelligent  Control 

•  A  comprehensive  literature  updated  on  the  topic  of  control  techniques  for  prosthetic  hands  was  conducted. 

•  Trajectory  planning,  kinematics  and  dynamics  of  a  prosthetic  hand  with  a  two -link  thumb  and  four  three- 
link  fingers  based  on  analytical  models  were  investigated. 

•  Hard  computing  or  control  techniques  such  as  feedback  linearization,  proportional -derivative  (PD), 
proportional-integral-derivative  (PID),  optimal  and  adaptive,  were  developed  for  the  14  degrees  of  freedom 
(DOFs),  five-fingered  smart  prosthetic  hand. 

•  Soft  control  techniques  such  as  neural  networks  (NN),  fuzzy  logic  (FL),  adaptive  neural-fuzzy  inference 
system  (ANFIS),  and  genetic  algorithm  (GA)  were  investigated. 

•  Finally,  the  fusion  of  hard  and  soft  control  techniques,  such  as  FL+PD  and  GA+PID,  was  developed  for  the 
five-fingered  smart  prosthetic  hand. 

•  Control  method  for  power  grasping  for  fully  actuated  hand 

•  Control  method  for  precision  grasping  for  fully  actuated  hand 

•  Optimal  design  for  underactuated  hand 

•  Control  method  for  underactuated  hand 

•  Comparison  study  of  underactuated  and  fully  actuated  hands 
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Goal  2B:  Embedded  Hierarchical  Real-Time  Systems 

•  Fast  prototyping  test  bed  with  common  software  tool  chain  and  processors 

•  Design  and  Implementation  of  real-time  control  strategies  on  the  test  bed. 

•  Real-time  operating  system  impact  on  the  embedded  control  with  multitasking 

•  Mass  storage  for  on-line  data  access  and  management  via  external  devices 

•  Investigation  of  modeling  and  simulation  for  the  embedded  control  system 
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Goal  3:  Robotic  Hand 


•  Implementation  of  a  working  five-fingered  prosthetic  hand  prototype  (Prototype  1)  based  on  a 
parallel  actuation  structure. 

•  Development  of  a  general  kinematic  synthesis  algorithm  and  solver  for  designing  kinematic 
structures  with  the  complexity  of  the  human  hand. 

•  Development  of  a  synthesis  theory  for  the  exact  design  of  articulated  systems  for  full  trajectories 
or  workspaces. 

•  Development,  setup  and  implementation  of  a  camera  system  to  recover  finger  pose  from  a  single 
camera. 
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Goal  4:  Biocompatibility  and  Signaling 

•  Development  of  a  co-culture  model  for  biocompatibility  and  other  tissue  engineering  studies  and 
applications 

•  Development  of  cell  models  of  peripheral  nervous  system  neural  cells  consisting  of  dorsal  root 
ganglion  neurons  and  Schwann  cells  for  biocompatibility  and  other  tissue  engineering  studies  and 
applications 

•  Characterization  of  cell  models  consisting  of  dorsal  root  ganglion  neurons  and  Schwann  cells 

•  Development  of  -smart”  biosensors  for  tissue  engineering  and  biocompatibility  studies  and 
applications 

•  Characterization  of  cytotoxicity  of  silicon  dioxide  nanoparticles  in  human  neural  and  non-neural  cells 

•  Elucidation  of  cell  death  mechanisms  underlying  cytotoxic  effects  induced  by  silicon  dioxide 
nanoparticles  in  human  astrocytoma  U87  (astrocytes -like)  cells 

•  Initiation  of  a  systematic  examination  of  putative  cytotoxicity  of  metallic  oxide  nanoparticles 
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REPORTABLE  OUTCOMES 


Goal  1:  EMG  Signal  Identification 

PI.  A.  Sebastian,  P.  Kumar,  M.  P.  Schoen,  -Evaluation  of  Filtering  Techniques  Applied  to  Surface  EMG  Data 
and  Comparison  Based  on  Hammerstein- Wiener  Models”,  in  The  10th  World  Scientific  and  Engineering 
Academy  and  Society  (WSEAS)  International  Conference  on  dynamical  systems  and  control  (CONTROL 
'll),  Iasi,  Romania,  July  1-3,  2011. 

P2.  A.  Sebastian,  P.  Kumar,  M.  P.  Schoen,  -Optimized  Spatial  Filter  Mask  Using  Genetic  Algorithm  And 
sEMG-Force  Modeling  Using  System  Identification”,  in  The  10th  World  Scientific  and  Engineering 
Academy  and  Society  (WSEAS)  International  Conference  on  dynamical  systems  and  control  (CONTROL 
'll),  Iasi,  Romania,  July  1-3,  2011. 

P3.  M.  Anugolu,  A.  Sebastian,  P.  Kumar,  M.  P.  Schoen,  A.  Urfer,  and  D.  S.  Naidu,  -Surface  EMG  Array 
Sensor  Based  Model  Fusion  Using  Bayesian  Approaches  For  Prosthetic  Hands,”  2009  ASME  Dynamic 
Systems  and  Control  Conference,  Hollywood,  California,  USA,  Oct.  12-14,  2009. 

P4.  C.  Potluri,  P.  Kumar,  M.  Anugolu,  A.  Urfer,  S.  Chiu,  D.  S.  Naidu,  and  M.  P.  Schoen,  — Ihquency  Domain 
Surface  EMG  Sensor  Fusion  for  Estimating  Finger  Forces,”  32nd  Annual  International  Conference  of  the 
IEEE  Engineering  in  Medicine  and  Biology  Society,  Buenos  Aires,  Argentina,  Aug.  31  -  Sept.  4,  2010. 

P5.  P.  Kumar,  A.  Sebastian,  C.  Potluri,  Y.  Yihun,  A.  Ilyas,  M.  Anugolu,  R.  Sharma,  J.  Creelman,  A.  Urfer,  D. 
S.  Naidu,  and  M.  P.  Schoen,  -Spectral  Analysis  of  sEMG  Signals  to  Investigate  Skeletal  Muscle  Fatigue,” 
50th  IEEE  Conference  on  Decision  and  Control  and  European  Control  Conference,  Orlando  (FL),  USA, 
December  12-15,  201 1  (Under  Peer  Review). 

P6.  P.  Kumar,  A.  Sebastian,  C.  Potluri,  A.  Urfer,  D.  S.  Naidu,  and  M.  P.  Schoen,  towards  Smart  Prosthetic 
Hand:  Adaptive  Probability  Based  Skeletal  Muscle  Fatigue  Model,”  32nd  Annual  International  Conference 
of  the  IEEE  Engineering  in  Medicine  and  Biology  Society,  Buenos  Aires,  Argentina,  Aug.  3 1  -  Sept.  4, 
2010. 

P7.  P.  Kumar,  C.  Potluri,  A.  Sebastian,  S.  Chiu,  A.  Urfer,  D.  S.  Naidu,  and  M.  P.  Schoen,  — ArAdaptive  Multi 
Sensor  Data  Fusion  with  Hybrid  Nonlinear  ARX  and  Wiener-Hammerstein  Models  for  Skeletal  Muscle 
Force  Estimation,”  The  14th  World  Scientific  and  Engineering  Academy  and  Society  (WSEAS) 
International  Conference  on  Systems,  Corfu  Island,  Greece,  July  22-24,  2010. 

P8.  P.  Kumar,  C.  Potluri,  A.  Sebastian,  S.  Chiu,  A.  Urfer,  D.  S.  Naidu,  and  M.  P.  Schoen,  Adaptive  Multi 
Sensor  Based  Nonlinear  Identification  of  Skeletal  Muscle  Force,  WSEAS  Transactions  on  Systems,  Issue 
10,  Volume  9,  October  2010,  pp.  1051-1062,  2010. 

P9.  P.  Kumar,  C.  Potluri,  M.  Anugolu,  A.  Sebastian,  J.  Creelman,  A.  Urfer,  S.  Chiu,  D.  S.  Naidu,  and  M.  P. 
Schoen,  — AHybrid  Adaptive  Data  Fusion  with  Linear  and  Nonlinear  Models  for  Skeletal  Muscle  Force 
Estimation,”  5th  Cairo  International  Conference  on  Biomedical  Engineering,  Cairo,  Egypt,  Dec.  16-18, 
2010. 

P10.  P.  Kumar,  C.  Potluri,  A.  Sebastian,  Y.  Yihun,  A.  Ilyas,  M.  Anugolu,  R.  Sharma,  S.  Chiu,  J.  Creelman,  A. 
Urfer,  D.  S.  Naidu,  and  M.  P.  Schoen,  — Alybrid  Adaptive  Multi  Sensor  Data  Fusion  for  Estimation  of 
Skeletal  Muscle  Force  for  Prosthetic  Hand  Control,”  The  2011  International  Conference  on  Artificial 
Intelligence,  ICAT1 1,  Las  Vegas,  Nevada,  USA,  July  18-21,  201 1. 

Pll.  P.  Kumar,  C.  H.  Chen,  A.  Sebastian,  M.  Anugolu,  C.  Potluri,  A.  Fassih,  Y.  S.  Yihun,  A.  Jensen,  Y.  Tang, 
S.  Chiu,  K.  Bosworth,  D.  S.  Naidu,  J.  Creelman,  A.  Urfer  and  M.  P.  Schoen,  — ArAdaptive  Hybrid  Data 
Fusion  Based  Identification  of  Skeletal  Muscle  Force  with  ANFIS  and  Smoothing  Spline  Curve  Fitting,” 
2011  IEEE  International  Conference  on  Fuzzy  Systems,  Taipei,  Taiwan,  June  27-30,  2011. 

P12.  P.  Kumar,  A.  Sebastian,  C.  Potluri,  A.  Urfer,  D.  S.  Naidu,  and  M.  P.  Schoen,  towards  Smart  Prosthetic 
Hand:  Adaptive  Probability  Based  Skeletal  Muscle  Fatigue  Model,”  32nd  Annual  International  Conference 
of  the  IEEE  Engineering  in  Medicine  and  Biology  Society,  Buenos  Aires,  Argentina,  Aug.  3 1  -  Sept.  4, 
2010. 

P13.  P.  Kumar,  A.  Sebastian,  C.  Potluri,  A.  Ilyas,  M.  Anugolu,  A.  Urfer,  and  M.  P.  Schoen,  — Aaptive  Finger 
Angle  Estimation  form  sEMG  Data  with  Multiple  Linear  and  Nonlinear  Model  Data  Fusion,”  The  10th 
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Goal  2A:  Intelligent  Control 

Portions  of  goal  2 A  have  been  accepted  in  one  book  chapter  (attachment  2A-1),  reviewed  in  one  international 
journal  paper  (attachment  2A-2)  and  four  international  peer-reviewed  conference  papers  (attachments  2A-3,  2A-4, 
2A-5,  and  2A-6);  one  poster  (attachment  2A-7)  have  been  presented  as  follows: 
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Conference  on  Systems,  Latest  Trends  on  Systems  (volume  II),  Corfu  Island,  Greece,  July  22-24,  pp.  405- 
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(CSSrll),  Corfu  Island,  Greece,  July  14-16,  2011.  (accepted)  (attachment  2A-5) 

6.  C.-H.  Chen  and  D.  S.  Naidu.  Fusion  of  fuzzy  logic  and  PD  control  for  a  five-fingered  smart  prosthetic 
hand.  In  Proceedings  of  the  2011  IEEE  International  Conference  on  Fuzzy  Systems  (FUZZ-IEEE  2011), 
Taipei,  Taiwan,  June  27-30,  2011.  (accepted)  (attachment  2A-6) 

7.  C.-H.  Chen  and  D.  S.  Naidu.  Fusion  of  fuzzy  logic  and  PD  control  for  a  five-fingered  smart  prosthetic 
hand,  in  Graduate  Student  Research  and  Creative  Excellence  Symposium,  Idaho  State  University, 
Pocatello,  Idaho,  USA,  April  8,  2011.  (Poster  presentation)  (attachment  2A-7) 


8.  1-  Amir  Fassih,  D.  Subbaram  Naidu,  Steve  Chiu,  and  Marco  P.  Schoen  ,  -Power  grasping  of  a  prosthetic 

hand  based  upon  virtual  spring-damper  hypothesis”,  IASTED  2010,  Massachusetts  ,  Cambridge. 


9.  2-  Amir  Fassih,  D.  Subbaram  Naidu,  Steve  Chiu,  and  Marco  P.  Schoen  ,  -Precision  grasping  of  a  prosthetic 

hand  based  on  virtual  spring  damper  hypothesis”,  CIBEC  2010,  Egypt,  Cairo. 
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Goal  2B:  Embedded  Hierarchical  Real-Time  Systems 

The  following  related  refereed  publications  were  produced: 

[1]  Chandrasekhar  Potluri,  Yimesker  Yihun,  Parmod  Kumar,  Jeff  Molitor,  Steve  Chiu,  D.  S.  Naidu,  S.  H. 
Mousavinezhad,  -sEMG  Based  Real  Time  Embedded  Force  Control  Strategy  for  a  Prosthetic  Hand  Prototype,” 
Accepted  for  publication  in  the  IEEE  International  Conference  on  Electro/Information  technology  (IEEE  EIT, 
05/2011). 

[2]  Chandrasekhar  Potluri,  Parmod  Kumar,  Jeff  Molitor,  Madhavi  Anugolu,  Alex  Jensen,  Kenyon  Hart,  and  Steve 
Chiu,  -Alulti-Level  Motor  Control  for  Prosthesis,”  in  the  proceedings  of  the  2010  International 

Conference  on  Embedded  Systems  and  Applications  (ESA,  07/2010). 

[3]  Chandrasekhar  Potluri,  Parmod  Kumar,  Madhavi  Anugolu,  Steve  Chiu,  Marco  Schoen,  Alex  Urfer,  and  D.  S. 
Naidu,  — ArANFIS  Based  Fuzzy  Controller  for  Prosthetic  Hand,”  in  the  proceedings  of  the  3rd  IEEE 
RAS-EMBS  International  Conference  on  Biomedical  Robotics  and  Biomechatronics  (BioRob,  09/2010). 

[4] .  H.  Karepey,  H.  Ahsan,  S.  Chiu,  and  A.  Perez-Gracia,  — B)  Reconstruction  for  Hand  Tracking  and  Estimation,” 
in  the  proceedings  of  the  2010  International  Conference  on  Image  Processing,  Computer  Vision,  and  Pattern 
Recognition  (IPCV,  07/2010). 

[5] .  J.  Manchala  and  S.  Chiu,  — ANodal  Feedback  Model  for  Wireless  Multi-Sensory  Data  Fusion,”  in  the 
proceedings  of  the  2010  International  Conference  on  Wireless  Networks  (ICWN,  07/2010). 

[6] .  P.  Kumar,  N.  Joshi,  A.  Jensen,  C.  Potluri,  M.  Schoen,  and  S.  Chiu,  -Genetic  Algorithm  Running  Time 
Optimization  Using  OpenMP  Parallel  Computing,”  in  the  proceedings  of  the  2010  International  Conference  on 
Parallel  and  Distributed  Processing  Techniques  and  Applications  (PDPTA,  07/2010). 
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Goal  3:  Robotic  Hand 


Manuscripts  published  or  in  press  (see  Appendices  -Goal3_l,  Goal3_2,  Goal3_3): 

Perez  Gracia,  A.,  -Synthesis  of  Spatial  RPRP  Closed  Linkages  for  a  Given  Screw  System”,  ASME  Journal  of 
Mechanisms  and  Robotics ,  201 1,  in  press. 

Simo-Serra,  E.,  Moreno-Noguer,  F.,  and  Perez-Gracia,  A.,  — Dagn  of  Non-anthropomorphic  Robotic  Hands  for 
Anthropomorphic  Tasks”,  accepted  at  the  ASME  International  Design  Engineering  Technical  Conferences,  IDETC 
201 7,  Washington  DC,  USA,  August  29-31,  2011. 

Crawford,  A.L.,  Molitor,  J.,  Perez  Gracia,  A.,  and  Chiu,  S.,  -Design  of  a  Robotic  Hand  and  Simple  EMG  Input 
Controller  with  a  Biologically-Inspired  Parallel  Actuation  System  for  Prosthetic  Applications”,  1st  International 
Conference  on  Applied  Bionics  and  Biomechanics  (ICABB),  Venice,  Italy,  October  14-16,  2010. 

Abstracts  published: 

Presentations: 

A.  Invited  oral  presentations: 

B.  Oral  presentation: 

C.  Poster  presentations: 

D.  Invitation  to  chair  and/or  organize  sessions  at  national/international  meetings: 

Perez-Gracia:  Symposium  co-organizer  at  the  2011  ASME  International  Design  Engineering  Technical  Conferences 
and  Computers  and  Information  in  Engineering  Conferences,  Washington  DC,  USA,  August  28-31,  2011. 
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Goal  4:  Biocompatibility  and  Signaling 


Our  research  has  been  very  productive:  to  date,  we  have  7  journal  articles,  1  review,  and  7  conference 

proceedings  papers  published  or  in  press,  published  7  abstracts,  and  given  3  oral  and  4  poster 

presentations  (see  lists  below  for  details). 

Journal  Articles  published  or  in  press: 

1.  Lai  JCK,  Ananthakrishnan  G,  Jandhyam  S,  Dukhande  VV,  Bhushan  A,  Gokhale  M,  Daniels  CK  & 
Leung  SW  (2010)  Treatment  of  Human  Astrocytoma  U87  Cells  with  Silicon  Dioxide  Nanoparticles 
Lowers  Their  Survival  and  Alters  Their  Expression  of  Mitochondrial  and  Cell  Signaling  Proteins. 
International  Journal  of  Nanomedicine  5:715-723. 

2.  Jaiswal  AR,  Bhushan  A,  Daniels  CK  &  Lai  JCK  (2010)  A  Cell  Culture  Model  for  Diabetic  Neuropathy 
Studies.  Journal  of  the  Idaho  Academy  of  Science  46(l):58-63. 

3.  Wong  YYW,  Jaiswal  AR,  Bhushan  A,  Leung  SW  &  Lai  JCK  (2010)  Further  Elucidation  of 
Neuroprotective  Properties  of  Astrocytoma  (Astrocytes -like)  Cells.  Journal  of  the  Idaho  Academy  of 
Science  46(l):52-57. 

4.  Patil  PP,  Lai  MB,  Leung  SW,  Lai  JCK  &  Bhushan  A  (2010)  Differential  Cytotoxic  Effects  of 
Magnesium  Oxide  Nanoparticles  on  Cisplatin-sensitive  and  Cisplatin-resistant  Leukemia  Cancer  Cells. 
Journal  of  the  Idaho  Academy  of  Science  46(l):70-75. 

5.  Wright  GL,  Lai  JCK,  Chan  AWK,  Minski  MJ,  Lim  L  &  Leung  SW  (2010)  Metallomic  Distribution  in 
Various  Regions  of  the  Brain  as  Influenced  by  Dietary  Intakes  and  Their  Implications.  In  Proceedings 
of  the  ISEIS  2010  International  Conference  on  Ecological  Informatics  and  Ecosystem  Conservation, 
August  27-29,  Beijing,  China,  published  in  Procedia  Engineering  01  (2010)  0159-0171. 

6.  Leung  SW,  Siddhanti  S,  Williams  B,  Chan  AWK,  Minski  MJ,  Daniels  CK  &  Lai  JCK  (2010)  Effects 
of  Diet  Intake  on  Metal  and  Electrolyte  Distributions  in  Various  Organs.  In  Proceedings  of  the  ISEIS 
2010  International  Conference  on  Ecological  Informatics  and  Ecosystem  Conservation,  August  27-29, 
Beijing,  China,  published  in  Procedia  Engineering  01  (2010)  0104-0108. 

7.  Leung  SW,  Wang  Y  &  Lai  JCK  (201 1)  Biomedical  Applications  of  Modified  Carbon  Glassy  Electrode 
Sensor  with  Nanoparticles  and  Dendrimers.  Sensors  &  Transducers  (201 1)  Volume  11,  Special  Issue, 
April  201  l,pp.  74-82. 


Review  Articles  published  or  in  press: 

1.  Bhushan  A,  Patil  PP,  Leung  SW  &  Lai  JCK  (2011)  Metallic  Nanoparticles  in  Cancer  Imaging  and 
Therapy.  In:  CRC  Biomedical  Engineering  Handbook,  4th  Edition,  CRC  Press  (in  press). 


Conference  Proceedings  Papers  published  or  in  press: 

1.  Jaiswal  A,  Wong  YYW,  Bhushan  A,  Daniels  C  &  Lai  JCK  (2010)  A  Noncontact  Co-Culture  Model  of 
Peripheral  Neural  Cells  for  Nanotoxicity,  Tissue  Engineering  and  Pathophysiological  Studies. 
Technical  Proceedings  of  the  2010  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010, 

Vol.  3  Chapter  8:  Environment,  Health  &  Safety,  Pages  527-531. 

2.  Wong  YYW,  Jaiswal  AR,  Dukhande  VV,  Bhushan  A,  Leung  SW  &  Lai  JCK  (2010)  Elucidation  of 
Neuroprotective  Properties  of  Astrocytoma  (Astrocytes-like)  Cells  in  Neural  Cell  Culture  Models  In 
Vitro :  Applications  in  Tissue  Engineering  and  Nanotoxicology.  Technical  Proceedings  of  the  2010 
NSTI  Nanotechnology  Conference  and  Expo  -  Nano  tech  2010,  Vol.  3  Chapter  8:  Environment,  Health 
&  Safety,  Pages  561-564. 

3.  Leung  SW,  Gao  W,  Gu  H,  Bhushan  A  &  Lai  JCK  (2010)  Chitosan  Membrane  in  Combinations  with 
Nanoparticles  and  Adriamycin  as  a  Treatment  to  Inhibit  Glioma  Growth  and  Migration.  Technical 
Proceedings  of  the  2010  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  Vol.  3 
Chapter  3:  Nano  for  Biotech,  Interfaces  &  Tissues,  Pages  206-209. 
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4.  Leung  SW,  Wang  Y,  Gu  H  &  Lai  JCK  (2010)  Biomedical  Applications  of  Modified  Carbon  Glassy 
Electrode  Sensor  with  Nanoparticles  and  Dendrimers.  Technical  Proceedings  of  the  2010  NSTI 
Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  Vol.  3  Chapter  1:  Bio  Sensors,  Diagnostics  & 
Imaging,  Pages  122-124. 

5.  Bhushan  A,  Patil  PP,  Bhardwaj  V,  Lai  MB,  Daniels  CK,  Leung  SW  &  Lai  JCK  (2010)  Cross 
Resistance  of  Magnesium  Oxide  Nanoparticles  to  Cisplatin  in  Leukemia  Cells:  Mechanistic  Studies.  In 
Seminar  Proceedings  of  Institution  of  Mechanical  Engineers  Seminar  on  Nanotechnology  in  Medicine 
and  Biotechnology,  October  2010,  London,  UK,  Chapt.  5,  pp.  1-4. 

6.  Leung  S  &  Lai  J  (2011)  Advanced  Dehydrogenase  Biofuel  Cell  Modified  with  Highly  Branched 
Polymers  and  Nanogold  Sol-Gel.  In  Proceedings  of  201 1  International  Conference  on  Agricultural  and 
Biosystems  Engineering  (ICABE  201 1).  Lecture  Notes  in  Information  Technology  Vols.  3-4,  pp.  41- 
44. 

7.  Leung  S,  Williams  B,  Lai  J,  Chan  A  &  Minski  M  (201 1)  Elemental  Analyses  in  Organ  Systems  in  a 
Small  Animal  after  Life-Long  Controlled  Diet.  In  Proceedings  of  201 1  International  Conference  on 
Agricultural  and  Biosystems  Engineering  (ICABE  2011).  Lecture  Notes  in  Information  Technology 
Vols.  3-4,  pp.  45-48. 


Abstracts  published: 

1.  Jaiswal  A,  Wong  YYW,  Bhushan  A,  Daniels  C  &  Lai  JCK  (2010)  A  Noncontact  Co-Culture 
Model  of  Peripheral  Neural  Cells  for  Nanotoxicity,  Tissue  Engineering  and 
Pathophysiological  Studies.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010, 
June  21-25,  2010,  Anaheim,  CA  (in  Abstracts  Volume). 

2.  Wong  YYW,  Jaiswal  AR,  Dukhande  VV,  Bhushan  A,  Leung  SW  &  Lai  JCK  (2010) 
Elucidation  of  Neuroprotective  Properties  of  Astrocytoma  (Astrocytes -like)  Cells  in  Neural 
Cell  Culture  Models  In  Vitro :  Applications  in  Tissue  Engineering  and  Nanotoxicology.  NSTI 
Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim,  CA  (in 
Abstracts  Volume). 

3.  Leung  SW,  Gao  W,  Gu  H,  Bhushan  A  &  Lai  JCK  (2010)  Chitosan  Membrane  in 
Combinations  with  Nanoparticles  and  Adriamycin  as  a  Treatment  to  Inhibit  Glioma  Growth 
and  Migration.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June  21-25, 
2010,  Anaheim,  CA  (in  Abstracts  Volume). 

4.  Leung  SW,  Wang  Y,  Gu  H  &  Lai  JCK  (2010)  Biomedical  Applications  of  Modified  Carbon 
Glassy  Electrode  Sensor  with  Nanoparticles  and  Dendrimers.  NSTI  Nanotechnology 
Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim,  CA  (in  Abstracts 
Volume). 

5.  Wright  GL,  Lai  JCK,  Chan  AWK,  Minski  MJ,  Lim  L  &  Leung  SW  (2010)  Metallomic 
Distribution  in  Various  Regions  of  the  Brain  as  Influenced  by  Dietary  Intakes  and  Their 
Implications.  ISEIS  2010  International  Conference  on  Environmental  Informatics,  Beijing, 
China,  August  27-29,  2010  (in  Abstracts  Volume). 

6.  Leung  SW,  Siddhanti  S,  Williams  B,  Chan  AWK,  Minski  MJ,  Daniels  CK  &  Lai  JCK  (2010) 
Effects  of  Dietary  Intake  on  Metal  and  Electrolyte  Distributions  in  Various  Organs.  ISEIS 
2010  International  Conference  on  Environmental  Informatics,  Beijing,  China,  August  27-29, 
2010  (in  Abstracts  Volume). 

7.  Bhushan  A,  Patil  PP,  Bhardwaj  V,  Lai  MB,  Daniels  CK,  Leung  SW  &  Lai  JCK  (2010)  Cross 
Resistance  of  Magnesium  Oxide  Nanoparticles  to  Cisplatin  in  Leukemia  Cells:  Mechanistic 
Studies.  Institution  of  Mechanical  Engineers  Seminar  on  Nanotechnology  in  Medicine  and 
Biotechnology,  October  2010,  London,  UK  (in  Program  &  Abstracts). 


Presentations: 

E.  Invited  oral  presentations: 
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1 .  Wright  GL,  Lai  JCK,  Chan  AWK,  Minski  M  J,  Lim  L  &  Leung  S W  (20 1 0)  Metallomic 
Distribution  in  Various  Regions  of  the  Brain  as  Influenced  by  Dietary  Intakes  and  Their 
Implications.  ISEIS  2010  International  Conference  on  Environmental  Informatics,  Beijing,  China, 
August  27-29,  2010. 

2.  Leung  SW,  Siddhanti  S,  Williams  B,  Chan  AWK,  Minski  MJ,  Daniels  CK  &  Lai  JCK  (2010) 
Effects  of  Dietary  Intake  on  Metal  and  Electrolyte  Distributions  in  Various  Organs.  ISEIS  2010 
International  Conference  on  Environmental  Informatics,  Beijing,  China,  August  27-29,  2010. 

3.  Bhushan  A,  Patil  PP,  Bhardwaj  V,  Lai  MB,  Daniels  CK,  Leung  SW  &  Lai  JCK  (2010)  Cross 
Resistance  of  Magnesium  Oxide  Nanoparticles  to  Cisplatin  in  Leukemia  Cells:  Mechanistic 
Studies.  Institution  of  Mechanical  Engineers  Seminar  on  Nanotechnology  in  Medicine  and 
Biotechnology,  October  2010,  London,  UK. 

F.  Poster  presentations: 

1.  Jaiswal  A,  Wong  YYW,  Bhushan  A,  Daniels  C  &  Lai  JCK  (2010)  A  Noncontact  Co-Culture 
Model  of  Peripheral  Neural  Cells  for  Nanotoxicity,  Tissue  Engineering  and  Pathophysiological 
Studies.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010, 
Anaheim,  CA. 

2.  Wong  YYW,  Jaiswal  AR,  Dukhande  VV,  Bhushan  A,  Leung  SW  &  Lai  JCK  (2010)  Elucidation 
of  Neuroprotective  Properties  of  Astrocytoma  (Astrocytes -like)  Cells  in  Neural  Cell  Culture 
Models  In  Vitro :  Applications  in  Tissue  Engineering  and  Nanotoxicology.  NSTI  Nanotechnology 
Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim,  CA. 

3.  Leung  SW,  Gao  W,  Gu  H,  Bhushan  A  &  Lai  JCK  (2010)  Chitosan  Membrane  in  Combinations 
with  Nanoparticles  and  Adriamycin  as  a  Treatment  to  Inhibit  Glioma  Growth  and  Migration. 
NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim,  CA. 

4.  Leung  SW,  Wang  Y,  Gu  H  &  Lai  JCK  (2010)  Biomedical  Applications  of  Modified  Carbon 
Glassy  Electrode  Sensor  with  Nanoparticles  and  Dendrimers.  NSTI  Nanotechnology  Conference 
and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim,  CA. 
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CONCLUSIONS 


The  EMG  group  of  this  project  worked  on  the  physical  classification  (ongoing),  classification  based  on  spatial  filters 
(ongoing),  development  of  sensors  (to  be  tested),  spatial  filtering  (completed),  fatigue  modeling  (completed),  spatial 
filters  with  fatigue  (completed),  motion  model  (on  going).  According  to  the  stated  timeline  and  milestones  of  the 
original  proposal,  we  are  well  on  track  and  making  good  progress  to  the  final  goals  of  Phase  II. 

A  comprehensive  literature  updated  on  the  topic  of  control  techniques  for  prosthetic  hands  was  conducted  and  one 
book  chapter  is  printed.  The  forward  and  inverse  kinematics  of  the  system  regarding  the  analytical  relationship 
between  the  angular  positions  of  joints  and  the  positions  and  orientations  of  the  end-effectors  (fingertips),  was 
obtained  using  a  desired  orientation  □  Dfor  three-link  fingers.  The  differential  kinematics  for  a  serial  7?-link  revolute- 
joint  planar  prosthetic  hand  was  derived  to  compute  the  angular  velocities  and  angular  accelerations  of  each  finger. 
Besides,  adaptive  neuro-fuzzy  inference  system  (ANFIS)  was  used  for  inverse  kinematics  of  three-link  fingers. 
Then,  the  cubic  polynomial  function  was  designed  to  track  the  orientation  in  order  to  calculate  inverse  kinematics 
and  differential  kinematics.  An  adaptive  control  strategy  was  developed  for  the  14  degrees  of  freedom  (DOFs),  five- 
fingered  smart  prosthetic  hand  with  unknown  mass  and  inertia  of  all  the  fingers.  The  simulations  of  the  resulting 
adaptive  controller  showed  good  agreement  between  the  reference  and  the  actual  trajectories.  Work  is  in  progress  for 
developing  an  adaptive/robust  controller  for  the  five  fingered  hand  with  14 -DOFs.  To  make  the  optimal  controller 
fast  acting,  the  performance  index  J  included  an  exponential  term.  The  simulation  results  demonstrated  a  30 -fold 
increase  in  fast  response  compared  to  ANFIS  or  GA  based  trajectory  planning.  Work  is  underway  to  extend  this 
methodology  to  five-fingered,  three-dimensional  prosthetic  hand.  A  hybrid  control  technique  combining  soft  control 
with  ANFIS  and  genetic  algorithm  (GA)  and  hard  control  with  proportional-integral-derivative  (PID)  was  presented 
for  a  five-fingered  smart  prosthetic  hand.  The  ANFIS  is  used  for  inverse  kinematics  and  GA  is  used  for  tuning  the 
PID  parameters  with  the  objective  of  minimizing  the  error  squared  between  desired  and  actual  angles  of  the  links  of 
the  fingers.  Simulation  results  for  all  the  five  fingers  with  GA-tuned  PID  controller  showed  superior  performance 
compared  to  the  PID  control  alone.  A  real-time  implementation  of  this  technique  on  a  prototype  of  a  prosthetic  hand 
is  planned  for  future  work.  For  a  five-fingered  smart  prosthetic  hand,  a  hybrid  control  technique  involving  soft 
control  with  ANFIS  and  fuzzy  logic  (FL)  and  hard  control  with  proportional -derivative  (PD)  was  presented.  The 
ANFIS  is  used  for  inverse  kinematics  and  FL  is  used  for  tuning  the  PD  parameters  with  two  input  layers  (error  and 
error  change)  using  7  triangular  membership  functions  and  49  fuzzy  logic  rules.  Simulation  results  with  FL -tuned 
PD  (hybrid)  controller  showed  superior  performance  compared  to  the  PD  and  PID  control  alone.  Work  is  underway 
to  extend  this  methodology  to  a  real-time  implementation  on  the  prototype  of  a  prosthetic  hand.  In  this  report  two 
type  of  hand  which  are  fully  actuated  and  underactuated  hand  studied.  Appropriate  control  methods  are  provided  for 
precision  and  power  grasping  for  fully  actuated  hand.  For  underactuated  hand  the  optimal  design  condition  derived 
and  tested.  Appropriate  controller  for  hand  movement  proposed  and  tested  through  numerical  simulation.  Two 
methods  of  power  grasping,  based  on  hilly  actuated  and  underactuated  hand  compared  and  the  results  were  close 
which  means  both  method  are  capable  of  providing  secure  grasp  for  prosthetic  hand. 

The  work  demonstrated  the  usability  of  a  fast  prototyping  test  bed  for  the  validation  and  verification  of  model-based 
controller  designs.  We  were  also  able  to  study  in  more  details  the  impact  of  implementation  on  the  embedded 
control,  including  the  use  of  CAN  and  RTOS.  Furthermore,  we  began  our  investigation  on  the  modeling  and 
simulation  of  our  embedded  control  through  collaboration  with  Johns  Hopkins  University4  s  Applied  Physics 
Laboratory.  Based  on  the  promising  test  results  obtained  in  the  two-  fingered  (Thumb-index)  prototype,  mechanical 
design  of  the  five-fingered,  full-dof  has  been  designed,  and  modeled  in  CAD  software  and  validated  through 
simulation.  At  this  time,  the  five-fingered  prosthetic  hand  prototype  (Prototype  1)  is  finished  in  its  prototype  form. 
The  actuators,  particularly  the  region  I  actuation  scheme,  are  also  implemented.  Using  this  actuation  scheme,  simple 
tasks,  such  as  grasping  spherical  objects,  have  been  tested.  Similarly,  the  force  at  the  tip  of  the  finger  while  grasping 
has  been  tested  using  FSR;  the  implementation  of  position  sensors  is  still  in  progress.  During  the  next  three  to  six 
months,  the  testing  scheme  will  be  developed  and  the  hill  testing  of  Prototype  1  will  be  finished.  The  results  will  be 
useful  to  assess  the  need  for  the  parallel  actuation,  and  also  as  an  input  in  the  design  of  Prototype  2.  For  the  design 
of  Prototype  2  we  are  developing  new  kinematic  synthesis  results.  A  general  algorithm,  already  implemented  and  in 
the  phase  of  being  refined,  will  allow  us  to  size  kinematic  chains  with  the  complexity  of  the  human  hand.  Newly 
developed  theory  will  also  allows  us  to  approximate  or  follow  exactly  whole  trajectories  of  fingertips  with  new 
designs.  This  is  complemented  with  a  thorough  study  of  actual  designs.  The  focus  of  the  Prototype  2  design  will  be 
on  thumb  and  palm  redesign,  and  on  the  underactuated/fully  actuated  duality  scheme.  An  important  tool  for  many  of 
these  activities  is  a  hand  motion  identification  and  tracking  system,  which  is  being  developed  with  good  results  and 
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will  be  ready  to  be  used  in  the  next  three  months.  This  system  will  be  used  not  only  on  the  synthesis  process,  but 
also  for  shedding  some  light  on  grasping  and  manipulation  issues.  Regarding  this  last  topic,  our  efforts  will  be 
focused  on  identifying  hand  synergies  and  limits  to  allow  us  to  lower  the  dimensionality  of  the  hand  motion  space. 

We  have  made  significant  progress  in  studies  specified  on  biocompatibility  and  signaling.  In  the  process  of  setting 
up  the  -Sensor  on  Cells  on  Skin  Model,”  we  have  identified  the  three  stages  we  need  to  undergo  to  achieve  this  goal: 
(i)  stage  one  involves  the  development  of  different  human  and  other  mammalian  cells  (including  nerve  and  skin 
cells)  in  co-culture;  (ii)  stage  two  requires  the  juxtaposition  of  cells  in  relation  to  the  membrane;  and  (iii)  stage  three 
involves  designing  the  sensor(s)  and  fitting  them  into  the  — sesor  on  cells  on  skin  model”.  We  have  accomplished 
stages  (i)  and  (ii)  by  developing  the  -eo-culture”  and  the  — on-contact”  cell  models  and  demonstrating  that  these 
models  can  be  useful  and  relevant  in  facilitating  studies  to  address  various  mechanistic  issues  in  biocompatibility 
and  other  tissue  engineering  and  nanotoxicity  research.  Furthermore,  we  have  developed  new  cell  models  of 
peripheral  nervous  systems  consisting  of  dorsal  root  ganglion  neurons  and  Schwann  cells.  These  models  are 
particularly  useful  for  elucidating  nerve  degeneration  and  regeneration.  We  are  currently  characterizing  these 
models  further.  Our  extensive  literature  survey  indicates  that  our  approaches  are  truly  novel  and  will  lead  to  further 
ground-breaking  discoveries  because,  as  we  have  already  discussed  above,  the  whole  important  issue  of  how  tissue 
and  cellular  inflammatory  responses  are  elicited  by  implant  material(s)  have  not  been  resolved  and  is  poorly 
understood.  By  way  of  heading  towards  completing  stage  (iii),  we  have  developed  several  -^siarf  ’  sensors  that  are 
pH  sensitive  and  self-assembling  with  nanoparticles.  Thus  far,  we  are  using  glassy  carbon,  gold,  or  silver  as  anchor 
material  for  the  sensor  electrode.  This  type  of  -^siart”  sensor  can  be  employed  to  detect  many  enzymatic  and 
chemical  reactions  known  to  occur  in  different  tissues/organs.  Thus,  we  are  focusing  on  the  development  of 
biosensors  for  tissue  engineering  and  biocompatibility  studies  and  related  biomedical  engineering  applications.  We 
also  recently  noted  that  some  of  the  sensors  we  have  designed  can  also  be  developed  into  fuel  cells  and  related 
applications.  One  important  but  as  yet  poorly  characterized  facet  of  biocompatibility  pertains  to  the  putative 
nanotoxicity  of  nanoparticles  and  other  nanomaterials  employed  to  fabricate  artificial  implants.  As  part  of  our 
systematic  studies  to  elucidate  the  cytotoxicity  of  nanoparticles  of  metallic  oxides  that  are  commonly  found  in 
composites  in  fabricating  such  implants,  we  have  further  characterized  the  effects  of  silicon  dioxide  nanoparticles  on 
human  neural  cells  and  normal  human  fibroblasts.  We  have  elucidated  some  of  the  cellular  and  molecular 
mechanisms  underlying  the  cytotoxicity  of  these  nanoparticles  on  human  neural  cells.  These  are  important  advances 
in  nanotoxicity  research  because  of  the  paucity  of  literature  on  the  cytotoxicity  of  nanoparticles  of  metallic  and  non- 
metallic  oxides  in  neural  cells  Thus,  our  findings  may  have  toxicological  and  other  pathophysiological  implications 
on  exposure  of  humans  and  other  mammalian  species  to  such  nanoparticles. 
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Abstract:  -  Electromyogram  (EMG)  signals  have  been  used  for  control  of  prosthetic  devices  in  the  past. 
However,  most  of  the  control  schemes  currently  used  are  based  on  threshold  values  of  the  EMG  signal  as 
reference  to  actuate  the  prosthetic  hand.  Using  such  a  control  strategy,  it  is  not  possible  to  capture  the 
underlying  dynamics  of  the  relationship  between  EMG  signals  and  the  intended  finger  movements  and  forces. 
We  propose  to  use  system  identification  based  dynamic  models  which  are  extracted  from  recorded  EMG 
signals  and  the  corresponding  finger  forces.  A  key  influence  on  the  resulting  quality  of  such  models  is  the 
filtering  of  the  EMG  signals.  This  paper  presents  a  thorough  analysis  of  spatial  filtering  and  other  filtering 
methods.  The  different  filters  are  compared  on  the  basis  of  the  EMG-finger  force  model  fit  values  obtained 
using  System  Identification  using  various  Non-Linear  Hammerstein-Wiener  models.  The  nonlinear  spatial 
filters  gave  better  fit  values  as  compared  to  the  standard  filtering  techniques. 


Key -Words:  -  Spatial  Filtering,  Hammerstein-Wiener,  Surface  Electromyogram  (sEMG). 


1  Introduction 

About  50%  of  the  people  who  need  upper  extremity 
prosthetics  do  not  use  them,  [1].  This  could  be  due 
any  one  of  the  common  following  reasons;  a) 
Dexterity,  b)  Comfort,  c)  Cost.  Despite  advances  in 
the  fields  of  manufacturing,  electronics,  signal 
processing,  sensor  design,  and  our  understanding  of 
biological  signals,  we  still  face  a  huge  challenge 
designing  a  prosthetic  device.  This  is  due  to  the  fact 
that  the  competition  of  such  a  device  is  with  one 
which  nature  has  gifted  us.  A  human  hand  is  without 
doubt  the  best  possible  design.  For  a  prosthetic 
hand,  to  encompass  all  of  the  human  hands  features 
and  capabilities  is  as  of  now  still  a  distant  reality. 
Electromyogram  (EMG)  signals  have  been  used  for 
quite  some  time  now  in  the  control  of  prosthesis. 
The  EMG  signal  is  a  small  voltage  signal  (in  mV), 
generated  by  skeletal  muscles.  This  signal  carries 
information  of  the  objective  the  user  would  like  to 
execute.  Using  surface  EMG  (sEMG)  signals, 
researchers  have  been  able  to  actuate  motors  on 
artificial  prosthetic  devices.  EMG  signals  can  be 
measured  using  intramuscular  electrodes,  needle 
electrodes,  or  by  placing  electrodes  on  the  surface  of 
the  skin.  The  purpose  for  which  the  sEMG  signal  is 
being  recorded  generally  determines  whether  it 
should  be  measured  within  the  muscle  using  needle 
electrodes  or  surface  measurement  using  electrodes 
on  the  skin  would  suffice.  If,  the  purpose  of 
recording  the  EMG  signal  is  to  look  for  diseases 
relating  to  a  particular  muscle,  one  might  be  better 
off  using  needle  electrodes.  On  the  other  hand,  if  it 
is  to  be  used  to  actuate  motors  on  a  prosthetic 


device,  which  a  user  might  want  to  take  off  and 
wear  without  the  need  of  medical  supervision, 
surface  EMG  would  suffice.  Measuring  the  EMG 
signal  on  the  surface  of  the  skin  is  also  less 
uncomfortable.  This  method  is  generally  suited  only 
for  superficial  muscles.  Even  though  recording 
(sEMG)  is  favorable,  it  is  plagued  with  many  issues, 
such  as  crosstalk  from  firing  of  multiple  motor 
points  in  the  vicinity  of  the  recording  location, 
which  can  cause  significant  corruption  of  the  signal 
at  that  site,  motion  artifacts,  and  poor  signal  if  the 
motor  unit  is  not  identified  correctly  etc.  Needle 
electrodes,  on  the  other  hand,  require  trained 
medical  professionals  for  appropriate  placement  of 
the  electrodes  in  the  muscle.  This  paper  investigates 
sEMG  signals  only  with  the  objective  to  extract 
intended  finger  forces.  The  sEMG-force  relationship 
is  modeled  using  Hammerstein-Wiener  models  to 
characterize  the  dynamics.  These  models  are 
required  to  characterize  the  controller  dynamics  for 
sEMG  based  prosthetic  hands.  We  look  to  overcome 
the  crosstalk  issue  in  the  measurement  by  using  an 
array  of  nine  (9)  sEMG  sensors,  and  utilizing  spatial 
filters  to  isolate  and  improve  the  quality  of  the 
signals  at  the  identified  site  for  EMG  recording. 
The  sensor  array  was  placed  on  the  motor  unit 
location,  which  was  identified  for  the  subject  using 
an  external  stimulator.  The  sensors  were  then  placed 
around  the  motor  unit  to  form  a  3x3  square  matrix. 
The  processed  signal  was  then  used  for  identifying 
various  dynamical  models  for  the  prediction  of  the 
force,  from  the  recorded  sEMG  signal,  that  was 
generated  during  various  voluntary  contractions  of 
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the  subjects’  hand.  A  comparison  of  the  various 
outcomes  (model  fit  values  computed)  of  the  system 
identification  process,  using  signal  processing 
techniques  stated  in  the  ISEK  [2]  guidelines  and  the 
spatially  filtered  signals  are  presented  in  this  paper. 
The  experiments  were  conducted  for  long  durations, 
in  order  to  analyze  the  effects  of  the  muscle  fatigue 
on  the  model  structures. 


2  Problem  Formulation 

EMG  signal  should  ideally  be  measured  at  a  motor 
unit.  A  motor  unit  (MU)  consists  of  an  a- 
motoneuron  in  the  spinal  cord  and  the  muscle  fiber 
it  innervates.  sEMG  signals  are  influenced  by 
multiple  factors,  some  of  which  are;  a)  shape  of  the 
volume  conductor,  b)  the  thickness  of  the 
subcutaneous  tissue  layers,  c)  tissue 
inhomogeneities,  d)  distribution  of  the  motor  unit 
territories  in  the  muscle,  e)  size  of  the  motor  unit 
territories,  f)  distribution  and  the  number  of  fibers  in 
the  motor  unit  territory,  g)  length  of  the  fibers,  h) 
spread  of  the  endplates  and  tendon  junctions  within 
the  motor  units,  and  i)  spread  of  the  innervations 
zones  and  tendon  regions  among  motor  units.  The 
type  of  detection  system  used  also  plays  an 
important  part  in  influencing  the  sEMG 
measurements.  Some  of  the  factors  which  need  to  be 
taken  into  account,  with  the  detection  systems,  are  a) 
skin  electrode  contact  (impedance,  noise),  b)  spatial 
filtering  for  signal  detection,  c)  inter-electrode 
distance,  d)  electrode  size  and  shape,  and  e) 
inclination  of  the  detection  system  relative  to  the 
muscle  fiber  orientation,  [3].  Since  sEMG  is  plagued 
by  a  multitude  of  issues,  as  pointed  out  in  this 
section,  one  cannot  approach  this  problem 
realistically  by  trying  to  account  for  each  of  the 
variables  in  the  measurement,  nor  oversimplify  the 
problem  at  hand  by  assuming  a  simple  linear  or  a 
non-linear  relation  between  the  sEMG  signal  and  the 
corresponding  finger  force  generated.  Hence  the 
approach  presented  in  this  paper  is  to  assume  a 
black-box  model  in  order  to  deduce  a  suitable 
relation  or  model  structure  for  the  two  signals.  This 
approach  has  been  found  to  be  of  merit  in  our 
previous  studies  and  has  been  been  reported  in  [4]  & 
[5]  to  yield  satisfactory  fits. 

3  Problem  Solution 

"Spatial  filtering"  is  broadly  defined  as  a  method 
which  computes  spatial  density  estimates  for  events 
that  have  been  observed  at  individual  locations. 
These  filters  are  used  when  there  is  no  a  priori  curve 
to  fit  to  a  data  series.  Instead,  it  relies  on  nearby  or 
adjacent  values  to  estimate  the  value  at  a  given 
point.  These  filters  take  out  variability  in  a  data  set 
while  retaining  the  local  features  of  data.  Spatial 
filtering  is  principally  associated  with  digital  image 
processing.  This  method  may  be  applied  to  almost 
any  data  m  the  form  of  a  grid.  The  most  common 
spatial  filters  are  the  low-pass  and  high-pass  spatial 
filters.  These  are  focal  functions  whose  operation  is 
determined  by  a  kernel  or  neighborhood 
ofNxN  cells  around  each  pixel  or  grid  position  [6]. 
Grid  cells  “covered”  by  a  kernel  are  multiplied  by 
the  matching  kernel  entry  and  then  the  weighted 
average  is  calculated  and  assigned  as  the  value  for 


the  central  cell,  G.  For  example,  an  asymmetric  3x3 
kernel  may  look  like  the  one  shown  in  Equation  (1), 
or  any  combination  of  the  weights.  Typically  a,  b 
are  positive  integers.  If  a=b= 1,  then  the  kernel 
provides  a  simple  smoothing  or  averaging  operation. 
The  weights  m  the  kernel  can  be  modified  for 
specific  cases  or  data  sets.  In  any  case  the  weighted 
average  is  divided  by  the  sum  of  the  elements  of  the 
kernel.  Filters  of  this  type  are  sometimes  referred  to 
as  low-pass  filters. 


a  a  a 


Symmetric  Kernel  =  \  a  b  a 


(1) 


a  a  a 


If  the  weights  in  the  kernel  is  similar  to  the  one  in 
Equation  (2)  and  a,  b ,  c  are  positive  integers,  and  if 
the  following,  b>a>c ,  is  true,  then  the  kernel  is 
described  as  a  Gaussian  filter  which  is  symmetric 
but  center- weighted. 

f  c  a  cl 


Symmetric  Kernel  =  \  a  b  a 


(2) 


c  a  c 


The  filtered  grid  value  ‘G’  of  an  m=NxN  kernel 
matrix,  with  Ct  set  of  coefficients  and  Pi  -  set  of 
source  grid  values,  is  calculated  as: 


m  ’ 

I  Cf 
i  =  1 

where  B  is  often  set  to  0.  B  is  a  bias  term  to  increase 
or  decrease  the  resulting  value  of  ‘G’.  This  kernel  is 
also  sometimes  referred  to  as  the  ‘filter  mask’. 
Linear  Spatial  Filtering:  Linear  spatial  filtering 
would  modify  the  sEMG  array  data  j  by  replacing 
the  value  at  each  location  with  a  linear  function  of 
the  values  of  nearby  data  points.  Moreover,  this 
linear  function  is  assumed  to  be  independent  of  the 
data  point  locations  (k,  /),  where  {k,  /)  are  the  indices 
of  the  data  points  in  fi  which  is  represented  by  a 
composite  data  matrix.  This  kind  of  operation  can  be 
expressed  as  convolution  or  correlation.  For  spatial 
filtering,  it's  often  more  intuitive  to  work  with 
correlation.  The  filtered  result  g(k,  l)  is  obtained  by 
centering  the  mask  over  pixel  (k,  l)  and  multiplying 
the  elements  of  /  with  the  overlapping  elements  of 
the  mask  and  then  adding  them  up.  In  other  words, 
the  objective  is  to  amplify  the  activity  of  motor 
unit/s  located  closest  to  the  recording  site  (ideally 
the  actual  motor  location  for  the  particular  finger) 
and  reducing  the  EMG  signal  generated  by  other 
motor  units  located  further  away  or  motor  nits  of 
other  fingers.  The  selectivity  of  surface  EMG 
recordings  can  be  increased  by  reducing  the 
electrode  size  (i.e.,  skin-electrode  contact  area  or 
inter-electrode  distance)  [7]  and/or  by  applying 
temporal  filters  [8].  More  recent  work  has  focused 
on  advances  in  the  design  of  surface  electrode  arrays 
[9-10]  to  extract  single  motor  unit  information  from 
sEMG.  A  large  number  of  traditional  [11-13]  and 
adaptive  [14]  linear  spatial  filters  have  been 
extensively  used  to  glean  more  information  out  of 
sEMG  signals  and  to  understand  it  much  better. 

For  this  paper,  the  experiments  were  carried  out  on  a 
healthy  male  subject  to  extract  dynamical  models 
describing  the  relationship  between  sEMG-force 
signals.  The  motor  points  were  located  using  a 
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Muscle  Stimulator,  manufactured  by  Rich-Mar 
Corporation  (model  number  HV  1100).  The  motor 
location  of  the  ring  finger  was  chosen  for  the 
experiments.  The  EMG  detection  system  used  was  a 
Delsys,  Bagnoli-16  channel  EMG  (DS-160,  S/N- 
1116).  The  sensors  used  for  measuring  the  sEMG 
action  potentials  were  three  pronged  DE  3.1 
differential  surface  electrodes.  The  subjects’  skin 
was  prepared,  according  to  the  ISEK  standards, 
before  the  sensors  were  placed  over  the  motor  point. 
The  electrodes  were  placed  along  the  muscle  fibers 
(Flexor  Digitorum  Superficiahs)  for  recording 
sEMG.  Multiple  sEMG  sensors  in  an  array 
configuration  were  mounted  on  and  around  the 
identified  motor  unit,  as  shown  in  Figure  1.  The 
subjects’  hand  was  placed  on  a  flat  surface;  the 
reference  electrode  was  placed  on  the  elbow  where 
there  is  no  sEMG  signal.  Sensor  CHI  was  placed  on 
the  identified  motor  unit  location.  CH2  and  CH3 
were  placed  along  the  muscle  fiber  in  front  and 
behind  CHI  respectively.  Channels  4-9  were  placed 
in  the  orientation  as  shown  in  Figure  1.  Nine 
different  experiments  were  conducted  and  the 
corresponding  sEMG  signal  was  measured 
simultaneously  from  all  the  nine  sensors.  The  force 
generated  by  the  subject’s  fingers,  for  a  given 
motion,  was  measured  using  a  stress  ball  with  a 
force  sensitive  resistor  (FSR)  mounted  on  it. 


Figure  1:  Experimental  Setup  -  Location  of  sEMG  sensors 


The  change  in  the  resistance  of  the  FSR  is  directly 
proportional  to  the  force  being  applied.  Figure  2 
shows  the  location  of  the  FSR  on  the  stress  ball. 
Experiments  1  and  2  were  used  to  check  for  any 
spurious  signals  that  might  be  recorded  due  to  the 
slight  angle  at  which  the  subjects’  hand  was  held. 
Experiments  3  to  6  were  done  using  a  stress  ball 
with  a  lesser  stiffness  as  compared  to  experiments  7, 
8  &  9. 


Figure  2:  Force  sensitive  Resistor  and  Thumb  Restrain 


Also,  a  thumb  restrain  was  used  for  experiments  5-9. 
The  thumb  restrain  is  showed  in  the  Figure  2.  The 
stress  ball  was  changed  as  we  were  also  interested  in 
looking  at  the  change  in  the  sEMG  signal  when 


fatigue  occurs,  and  also  how  it  would  affect  the 
modeling  for  the  relation  between  force-sEMG 
using  System  Identification  (SI). 

The  linear  spatial  filters  tested  in  this  paper  for 
isolating  the  motor  unit  action  potentials  (MUAPs) 
are;  1)  Longitudinal  Single  Differential  (LSD),  2) 
Transverse  Single  Differential  (TSD),  3) 
Longitudinal  Double  Differential  (LDD),  4) 
Transverse  Double  Differential  (TDD),  5)  Normal 
Double  Differential  (NDD),  6)  Inverse  Binomial 
(IB2)  and  7)  Inverse  Rectangular  (IR)  Filter.  The 
masK  of  these  filters  and  the  corresponding  resultant 
equations  on  application  of  the  mask  to  the  grid  data 
obtained  from  the  sEMG  array  arrangement  are 
given  below. 

EMG  Array  Information,  Spatial  Filter  Mask 
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LSD  Equation  =  sEMGl-sEMG2;  TSD  Equation  = 
sEMGl-sEMG5.  We  can  similarly  deduce  the 
equations  for  the  other  spatial  filters. 
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In  this  paper,  nonlinear  spatial  filters  have  also  been 
discussed  these  have  been  reported  in  literature  [15]. 
These  are  1)  1-D  Nonlinear  Transverse  spatial  filter 
(NLT),  2)  1-D  Nonlinear  Longitudinal  spatial  filter 
(NLL),  3)  2-D  Nonlinear  spatial  filter  in  Two- 
Orthogonal  Directions  (NLTOD)  and  4)  Nonlinear 
spatial  filter  in  All  Four  possible  Directions 
(NLAFD).  The  nonlinear  spatial  filters  use  the 
Teager-Kaiser  Energy  (TKE)  Operator,  [16].  This 
technique  is  a  threshold  ‘energy’  based  approach 
where  outliers  are  first  detected  and  then  replaced  by 
their  estimated  values.  The  nonlinear  spatial  filters 
with  the  TKE  operator  incorporated  are  given  as 
follows;  The  general  form  of  nonlinear  spatial  filter 
using  the  (TKE)  operator  is  given  in  Equation  (4); 

X¥[x(n)]  =  (n)  -  x(n +  \)x(n -X)  (4) 

a)  1-D  Nonlinear  Transverse  Spatial  Filter  (NLT); 
Equation  (5) 

2 

T 3  m  [x(m> n)]  =  x  n)  -  x(m  -  1,  n)x(m  +  1,  n)  (5) 

b)  1-D  Nonlinear  Longitudinal  Spatial  Filter  (NLL); 
Equation  (6) 

2 

^ d  n  [x(m?  «)]  =  x  n)  ~  x(m,  n  -  l)x(m,  n  +  1)  (6) 


Page  188  of  531 


c)  Nonlinear  Spatial  Filter  in  Two  Orthogonal 
Directions  (NLTOD):  Equation  (7) 

^ d,2  [x(m’ n ^ d,2m  [ x ")]  +  ^  (7) 

2 

=2x  (m,  n)  -  x(m  -  1,  n)x(m  +  1,  w)  —  x(m,  «  -  l)x(m, «  +  1) 

d)  Nonlinear  Spatial  Filter  in  all  Four  Directions 
(NLAFD):  Equation  (8) 

2 

d  4  [x(m, «)]  =  4x  ( m ,  n )  -  x(m  -  1,  «)x(m  +  1, «) 

(8) 

-x(m,  n  -  1  )x(m,  n  +  1)  -  x(m  -  1  ,n  +  1  )x(m  +  1,  n  -  1) 

-x(m  -  1,  n  -  1  )x(m  +  1,  «  +  1) 


4  Simulation  Results 

Plots  of  the  raw  data  gathered  are  shown  in  Figure  3. 
The  plots  shown  in  Figure  3  represent  the  raw  data 
collected  from  three  channels  of  the  grid,  Channels 
1,  4  &  5. 

(CHI) 


Samples 


Figure  3:  sEMG  Raw  Signal  Channels  1,  4  and  5 


The  data  shown  above  is  for  comparison  of  the 
sEMG  obtained  from  various  location  of  the 
subject’s  arm.  Notice  the  change  especially  in  the 
amplitude  of  the  sEMG  at  Channel  1,  4  &  5.  Figure 
4  shows  the  plot  of  the  filtered  sEMG  using  the 
various  spatial  filters.  In  addition  to  these  1 1  spatial 
filters,  the  sEMG  data  was  also  filtered  using  4  other 
filters  -  Bessel,  Butterworth,  Chebyshev  Type  I  and 
Chebyshev  Type  II  filters.  The  filter  characteristics 
of  these  4  filters  were  in  accordance  to  the  ISEK 
standards.  The  relation  of  sEMG-Force  was 
modeled  using  Non-Linear  Hammerstein- Wiener 
models.  This  modeling  method  has  been  proven  to 
work  in  the  past  [4],  [5]  in  assuming  a  black-box 
model  structure  for  the  system.  In  order  to  model  the 
sEMG-Force  relation,  the  filtered  data  from  the 
various  experiments  was  split  into  various  four  (4) 
time  windows. 

Equation  9  describes  the  general  Hammerstein- 
Wiener  model  structure: 

B ;  j (a) 

=  &(0  =  -tttvK0,  y{t)  =  h(x(t)),  (9) 

FjA <7) 

where,  w(t)  and  b(t )  are  internal  variables,  w(t)  has 
the  same  dimensions  as  u{t)  -  input,  and  x{t)  has  the 
same  dimensions  as  y(t)  -  output.  g()  and  h()  are  the 
input  and  output  non-linearity  functions 
respectively.  B(q)  and  F(q)  are  regression 


polynomials.  The  model  fit  values  are  computed 
using  Equation  (10) 
as  follows; 


1-1  \y  -  y 

fit  =  1  0  0  *  |,  11 

L y-y\\ 


(10) 


where,  y  is  the  estimated  output  by  the  model. 


Figure  4:  Spatially  Filtered  sEMG  at  Motor  unit  Ring  Finger  a) 
NLT,  b)  NLL,  c)  NLTOD,  d)  NLAFD,  e)  LSD,  f)  TSD,  g) 
LDD,  h)  TDD,  i)  NDD,  j)  IB2,  k)  IR 


The  time  windows  used  for  estimation  and 
validation  of  the  models  were  called  ‘ze’  and  ‘zv* 
respectively.  6 ze  ’  contained  8000  sample  points  and 
‘zv*  contained  data  points  shifted  by  2000  sample 
points.  For  example,  if  ‘ze’  was  a  time  window 
between  2-6  seconds  i.e.  samples  4000-12000,  then 
‘zv*  was  between  3-7  seconds  i.e.  6000-14000 
samples.  Thus  the  Hammerstein- Wiener  method 
uses  ‘ze  ’  to  estimate  the  model  structure  and  based 
on  this  information  predicts  the  next  2000  sample 
points.  The  data  was  filtered  using  the  various  filters 
mentioned  in  the  previous  sections.  We  would  like 
to  stress  once  again  the  point  that  the  force  was 
varied  randomly  and  the  subject  was  in  no  way 
trying  to  achieve  maximum  voluntary  contractions 
during  each  cycle.  A  cycle  is  defined  as  the  subject 
starting  without  any  force  on  the  stress  ball, 
squeezing  it  (to  any  force  level)  and  then  going  back 
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to  no  force.  The  subject  has  to  keep  the  finger  in 
contact  with  the  force  ball  throughout  this  cycle. 

A  plot  of  the  variations  in  force  achieved  for  3 
experiments  is  shown  in  Figure  5.  The  Matlab™ 
code  for  the  Hammer  stein- Wiener  model  is: 
nlhw(ze,  [na  nb  nk],  ....,  ...).  The  modeling  was 
carried  out  by  varying  na  -  the  number  of  past  output 
terms  used  to  predict  the  current  output,  n,-  the 
number  of  past  input  terms  used  to  predict  the 
current  output  and  nk-  the  delay  from  input  and 
output  in  terms  of  the  number  of  samples  for  the 
various  Hammerstein- Wiener  models. 


Samples 

Figure  5:  Force  levels  experiment  3  (blue),  5(red)  &  7(green) 


models  for  the  other  experiments  too.  Another  set  of 
results  which  gave  very  high  values  of  fit  was  for 
the  later  time  windows.  This  case  was  especially 
interesting  as  the  subject  had  fatigued  due  to  the 
repetitive  experiments,  but  the  Hammerstein- Wiener 
models  did  successfully  capture  the  variations  in  the 
sEMG  signal.  The  Hammer  stein- Wiener  models 
performed  very  well  and  we  obtained  fit  values  in 
the  high  sixties. 


Figure  7:  Measured  data  (black)  a)  Bessel  (blue)  56.36%  and  b) 
NLTO  (red)  60.13%  for  exp  3  time  window  2-7seconds. 


42  models  with  variations  in  na  and  «fewere  tested 
while  the  value  of  nk  was  kept  as  1.  The  total 
number  of  models  estimated  were  15  (filter  types)  x 
4  (time  windows)  x  42  models  per  time  window  x  4 
experiments  =  10,080  models.  This  paper  does  not 
list  all  these  models  but  identifies  and  reports  only 
the  significant  results  of  the  analysis. 


1  6  11  16  21  26  31  36  41 


Figure  6:  Fit  %  for  Hammerstein- Wiener  Models  -  Using 
Bessel  and  NLTO  Filters  (time  window  2-7  sec,  exp3) 


Figure  8  shows  the  model  fit  values  obtained  for 
these  later  data  windows.  As  can  be  seen  from  the 
plots,  contrary  to  our  expectations  the  Hammerstein- 
Wiener  Models  performed  very  well  and  produced 
very  good  fit  values. 


Figure  8:  Fit  Values  Obtained  for  Different  Hammerstein- 
Wiener  Models  -  Bessel,  NLTO  NLT,  TDD,  NDD,  NLAFD 
Filters  (time  window  20-25  sec,  exp5) 


As  an  example,  for  experiment  3,  for  time  window 
of  2-7  sec,  the  fit  values  obtained  by  varying  na 
between  2-7  and  ^between  3-9,  are  shown  in 
Figure  6.  The  fit  values  ranged  from  33-56%  for  the 
Bessel  filter  and  from  20-60%  for  the  NLTO  filter. 
The  best  values  obtained  for  this  time  window  were 
56.36%  and  60.13%  by  the  Bessel  and  the  NLTO 
filters  respectively.  The  model  output  plot  for  these 
values  is  shown  in  Figure  7.  The  large  variation  in 
the  model  fit  values  can  be  attributed  to  the  fact  that 
the  two  data  sets  have  poor  correlation  between  one 
another.  The  other  filters  used  also  predict  the  future 
variations  in  force  but  the  fit  percentages  were  in  the 
range  of  30-48%.  A  key  objective  of  this  research  is 
help  develop  a  control  regime  which  in  not  based  on 
threshold  values  of  force  can  incorporate  the 
dynamics  in  the  force.  This  would  help  to  control 
the  response  of  the  artificial  limb  to  be  closer  to  that 
of  the  actual  hand.  Similarly  we  tested  various 


Figure  9  shows  the  model  output  plots  for  some  of 
the  filters  mentioned  in  Figure  8. 


Figure  9:  Measured  Data  Black  -  a)  TDD  (70.28%),  b)  NDD 
(68.09%),  c)  NLT  (68.26%),  d)  NLAFD  (72.16%),  Filters 
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4  Conclusion 

The  Hammer  stein- Wiener  models  worked  very  well 
in  capturing  the  dynamics  of  the  force  levels  for  the 
various  experiments  conducted.  This  method  of 
modeling  could  help  in  improving  the  control  over 
the  motors  used  in  prosthetic  devices  to  mimic  the 
actual  changes  force  levels  in  a  real  hand.  This 
method  also  performed  very  well  in  the  scenarios 
where  the  subject  did  fatigue  but  the  affects  were 
successfully  modeled  by  the  Hammer stein-Wiener 
models.  The  nonlinear  and  linear  spatial  filters 
(TDD,  NDD  and  NLT,  NLTO  and  NLAFD)  did 
outperform  the  other  filtering  methods  used 
especially  for  the  later  time  windows.  The  only 
other  filter  which  had  a  comparable  performance  to 
the  spatial  filters  was  the  Bessel  filter.  Further 
investigation  into  reducing  the  wide  range  of  the  fit 
values  obtained  needs  to  be  performed.  One  of  the 
possible  methods  to  pursue  would  be  to  use  Genetic 
Algorithm  to  optimize  the  model  parameters  na  and 

nb  and  also  the  number  of  iterations  used  for  the 

modeling  of  sEMG-force  levels.  One  of  the  possible 
reasons  for  poor  fit  values  could  also  be  attributed 
to  the  model  trying  to  over-fit  the  data  sets. 
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Abstract:  -  This  paper  presents  the  investigation  of  the  use  of  System  Identification  (SI)  for  modeling  sEMG- 
Finger  Force  relation  in  the  pursuit  of  improving  the  control  of prosthetic  hands.  Finger  force  and  sEMG  data 
is  generated  by  having  the  subject  perform  a  number  of  random  motions  of  the  ring  finger  to  simulate  various 
force  levels.  Post-processing  of  the  sEMG  signal  is  performed  using  spatial  filtering.  The  linear  and  nonlinear 
spatial  filters  are  compared  based  on  the  ‘kurtosis  ’  improvements  and  also  are  based  on  the  fit  values  of  the 
models  obtained  using  system  identification.  Some  of  the  spatial  filter  masks  are  optimized  using  a  Genetic 
Algorithm  (GA)  for  the  constrained  and  unconstrained  cases.  The  resulting  model  fit  value  is  utilized  to  serve 
as  the  cost  function.  The  results  are  compared  to  the  reported  filter  mask  values  in  the  literature.  The 
unconstrained  GA  based  filter  mask  values  and  in  some  instances  the  constrained  GA  based  mask  values 
performed  better  than  the  filter  masks  reported  in  literature. 


Key-Words:  -  Spatial  Filtering,  System  Identification,  Surface  Electromyogram,  Array  Sensor. 


1  Introduction 

In  the  United  States  there  are  approximately  1.7 
million  people  living  with  limb  loss  [1].  It  is 
estimated  that  one  out  of  every  200  people  in  the 
U.S.  has  had  an  amputation  [2].  An  ideal  prosthetic 
hand  has  to  be  dexterous;  easy  to  manufacture,  must 
use  little  power  and  at  the  same  time,  must  be  of  low 
cost.  Building  such  a  prosthetic  hand,  which  can 
mimic  the  entire  gamut  of  motions  and  have  the 
functionality  and  dexterity  of  a  human  hand  exactly, 
has  eluded  researchers  so  far.  One  of  the  main 
strategies  used  in  making  a  prosthetic  hand  “user 
friendly”  is  to  use  the  electromyogram  (EMG) 
signal  to  control  a  prosthetic  device.  An  EMG  signal 
is  a  small  voltage  signal  (in  mV)  which  is  generated 
by  skeletal  muscles.  EMG  is  measured  using  fine 
wire  intramuscular  electrodes,  needle  electrodes,  or 
on  the  surface  of  the  skin  over  the  motor  point  using 
surface  electrodes.  Surface  electrodes  are  quick  and 
easy  to  apply,  do  not  need  medical  supervision, 
cause  minimal  discomfort  and  are  generally  suited 
for  superficial  muscles.  Needle  electrodes  on  the 
other  hand  require  trained  medical  professionals  for 
appropriate  placement  within  the  muscle.  This  paper 
looks  only  at  surface  electromyogram  signals 
(sEMG).  In  the  past,  multiple  methods  have  been 
investigated  for  extracting  useful  information  from 
EMG  signals.  Some  of  these  methods  include 
employing  low-pass  or  band-pass  filtering;  others 
have  also  used  notch  filtering  to  remove  power  line 
noise,  [3].  Whitening  filters  can  increase  the  quality 
of  the  amplitude  estimates  of  the  sEMG  signals,  [4]. 
Other  methods  include  Markov  models,  [5]  and 


fuzzy  logic  control,  [6]  -  for  classification  of  EMG, 
and  wavelet  processing,  [7].  Currently,  the  accepted 
standardized  method  on  how  EMG  signals  must  be 
recorded  and  analyzed  is  set  by  the  International 
Society  of  Electromyography  and  Kinesiology 
(ISEK)  [8].  sEMG  signals  are  influenced  by 
multiple  factors,  some  of  which  are;  a)  shape  of  the 
volume  conductor,  b)  the  thickness  of  the 
subcutaneous  tissue  layers,  c)  tissue 
inhomogeneities,  d)  distribution  of  the  motor  unit 
territories  in  the  muscle,  e)  size  of  the  motor  unit 
territories,  f)  distribution  and  the  number  of  fibers  in 
the  motor  unit  territory,  g)  length  of  the  fibers,  h) 
spread  of  the  endplates  and  tendon  junctions  within 
the  motor  units,  and  i)  spread  of  the  innervations 
zones  and  tendon  regions  among  motor  units.  The 
type  of  detection  system  used  also  plays  an 
important  part  in  influencing  the  sEMG 
measurements.  Some  of  the  factors  which  need  to  be 
taken  into  account,  with  the  detection  systems,  are  a) 
skin  electrode  contact  (impedance,  noise),  b)  spatial 
filtering  for  signal  detection,  c)  inter-electrode 
distance,  d)  electrode  size  and  shape,  and  e) 
inclination  of  the  detection  system  relative  to  the 
muscle  fiber  orientation,  [9]. 

2  Problem  Formulation 

As  multiple  factors  influence  sEMG,  it  is  impossible 
to  account  for  each  of  these  variables  while 
modeling  its  relation  to  the  force  generated.  Also, 
oversimplification  of  the  problem  at  hand  by 
neglecting  some  variables,  which  is  a  widely  used 
common  assumption,  will  cast  doubt  over  the 
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usefulness  of  the  results  of  modeling.  Our  approach 
is  to  assume  a  black-box  model  in  order  to  deduce  a 
suitable  relation  or  model  structure  for  the  two 
signals.  Here,  the  modeling  of  the  sEMG-finger 
force  relation  is  not  based  on  root-mean-square,  or 
average  values  of  the  sEMG  signal,  hence,  we 
facilitate  the  capture  of  the  dynamical  changes  in  the 
force  levels.  Our  approach  of  using  Hammerstein- 
Wiener  models  has  been  found  to  be  of  merit  in  our 
previous  studies  [10,  1 1]  to  yield  satisfactory  fits. 


3  Problem  Solution 

Spatial  Filtering 

The  data  collected  is  from  a  nine  (3x3)  sensor  array 
and  rather  than  analyzing  the  data  only  at  the  motor 
unit,  we  consider  using  spatial  filters.  Spatial 
filtering  is  a  very  attractive  choice  as  it  can  be  used 
to  either  amplify  the  signal  at  the  motor  unit  or 
extract  useful  information  from  the  entire  grid. 
’’Spatial  filtering”  is  broadly  defined  as  a  method 
which  computes  spatial  density  estimates  for  events 
that  have  been  observed  at  individual  locations. 
These  filters  are  used  when  there  is  no  a  priori  curve 
to  fit  to  a  data  series.  Instead,  it  relies  on  nearby  or 
adjacent,  values  to  estimate  the  value  at  a  given 
point.  The  most  common  spatial  filters  are  the  low- 
pass  and  high-pass  spatial  filters.  These  are 
focal  functions  whose  operation  is  determined  by  a 
kernel  or  neighborhood  of  NxN  cells  around  each 
pixel  or  grid  position  [12].  Grid  cells  “covered”  by  a 
kernel  are  multiplied  by  the  matching  kernel  entry 
and  then  the  weighted  average  is  calculated  and 
assigned  as  the  value  for  the  central  cell,  G.  For 
example,  an  asymmetric  3x3  kernel  may  look  like 
the  one  shown  in  Equation  (1),  or  any  combination 
of  the  weights.  Typically  a,  b  are  positive  integers. 
If  a=b= 1,  then  the  kernel  provides  a  simple 
smoothing  or  averaging  operation.  Filters  of  this 
type  are  sometimes  referred  to  as  low-pass  filters. 


Symmetric  Kernel 


a  a  a 
aba 


(1) 


a  a 


a 


The  filtered  grid  value  ‘G’  of  an  m=NxN  kernel 
matrix,  with  Ct  set  of  coefficients  and  Pt  -  set  of 
source  grid  values,  is  calculated  as; 


G  = 


(2) 


Where,  B  is  often  set  to  0.  B  is  a  bias  term  to 
increase  or  decrease  the  resulting  value  of  ‘G’.  This 
kernel  is  also  sometimes  referred  to  as  the  'filter 
mask \  The  linear  spatial  filters  tested  in  this  paper 
are;  1)  Longitudinal  Single  Differential  (LSD),  2) 
Transverse  Single  Differential  (TSD),  3) 
Longitudinal  Double  Differential  (LDD),  4) 
Transverse  Double  Differential  (TDD),  5)  Normal 
Double  Differential  (NDD),  6)  Inverse  Binomial 
(IB2)  and  7)  Inverse  Rectangular  (IR)  Filter.  The 
mask  of  these  filters  and  the  corresponding  resultant 
equations  on  application  of  the  mask  to  the  grid  data 


obtained  from  the  sEMG  array  arrangement  are 
given  below. 

EMG  Array  Information,  Spatial  Filter  Mask 
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0  0  0 


Result  Equation:  —  sEMG2  +  sEMGl 
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LSD  Equation  =  sEMGl-sEMG2;  TSD  Equation  = 
sEMGl-sEMG5 ;  we  can  similarly  deduce  the 
equations  for  the  other  spatial  filters. 
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In  this  paper,  some  nonlinear  spatial  filters  have  also 
been  discussed,  these  have  been  reported  in  the 
literature  [13],  these  are  1)  1-D  Nonlinear 

Transverse  spatial  filter  (NLT),  2)  1-D  Nonlinear 
Longitudinal  spatial  filter  (NLL),  3)  2-D  Nonlinear 
spatial  filter  in  Two-  Orthogonal  Directions 
(NLTOD)  and  4)  Nonlinear  spatial  filter  in  All  Four 
possible  Directions  (NLAFD).  The  Nonlinear 
Spatial  Filters  use  the  Teager-Kaiser  Energy  (TKE) 
Operator  [14].  This  technique  is  a  threshold  'energy’ 
based  approach  where  outliers  are  first  detected  and 
then  replaced  by  their  estimated  values.  General 
Form  of  Nonlinear  Spatial  Filter  using  the  Teager- 
Kaiser  (TKE)  operator  is  given  in  equation  (4), 
where  x(n)  denotes  the  location  of  the  sensor  in  the 
grid; 

XF[-X(^)]  -  (n)  -  x(n  +  \)x(n  - 1)  (4) 

a)  1-D  Nonlinear  Transverse  Spatial  Filter  (NLT); 
equation  (5) 

2 

d  m  [x(m> n)]  -  x  (m->  n)  ~  x(m  -  1,  n)x(m  +  1,  n)  (5) 

b)  1-D  Nonlinear  Longitudinal  Spatial  Filter  (NLL); 
equation  (6) 

2 

T d  n  \x(m,  n)]  =  x  (m,  n)  -  x(m,  n  -  n  +  1)  (6) 

c)  Nonlinear  Spatial  Filter  in  Two  Orthogonal 
Directions  (NLTOD);  equation  (7) 

^ d,2  M m ’  W)1  =  ^ d,2m  M m ’  W)1  +  ^ d,n  M m ’  n »  (7) 

2 

=2x  (m,  n )  -  x(m  -  1,  n)x(m  +  1,  n)  -  x(m,  n  -  1  )x(m,  n  +  1) 

d)  Nonlinear  Spatial  Filter  in  all  Four  Directions 
(NLAFD);  equation  (8) 

2 

VF  t  a  [x(m,  n)]  =  4x  (m,  n )  -  x(m  - 1,  n)x(m  +  1,  n) 
a (8) 

-x(m,  n  - 1  )x(m,  n  +  1)  —  x(m  —  l,n  +  1  )x(m  + 1,  n  —  1) 

-x(m  - 1  ,n  —  1  )x(m  +  1, n  +  1) 
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The  experiments  were  carried  out  on  a  healthy  male 
subject.  The  ring  finger  motor  point  was  located 
using  a  Muscle  Stimulator,  manufactured  by  Rich- 
Mar  Corporation  (model  number  HV  1100).  The 
EMG  detection  system  used  was  a  Delsys,  Bagnoli- 
16  channel  EMG  (DS-160,  S/N-1116).  The  sensors 
used  for  measuring  the  surface  EMG  action 
potentials  were  three  pronged  DE  3.1  differential 
surface  electrodes.  The  material  for  the  contacts  of 
the  electrode  is  99.9%  pure  silver,  the  contacts  are 
10mm  long,  1mm  in  diameter  and  spaced  10mm 
apart.  The  subjects’  skin  was  prepared,  according  to 
the  ISEK  standards,  before  the  sensors  were  placed 
over  the  motor  point.  The  electrodes  were  placed 
along  the  muscle  fibers  (Flexor  Digitorum 
Superficialis)  for  recording  surface  EMG.  The 
reference  electrode  was  placed  on  the  elbow  where 
there  is  no  sEMG  signal.  Nine  different  experiments 
were  conducted  and  the  corresponding  sEMG  signal 
was  measured  simultaneously  from  all  the  9  sensors. 
The  force  generated  by  the  subject’s  fingers,  for  a 
given  motion,  was  measured  using  a  stress  ball  with 
a  force  sensitive  resistor  (FSR)  mounted  on  it. 
Figure  1  shows  the  location  of  the  FSR  on  the  stress 
ball. 

System  Identification 

Equation  9  describes  the  Hammerstein- Wiener 
model  structure: 


Mf)  = 


b(t)  = 


FjM) 


w(0,  y(t)  -  h(x(t)\  (9) 


where,  w(t)  and  b(t)  are  internal  variables,  w(t)  has 


the  same  dimensions  as  u(t )  -  input,  and  x(t)  has  the 
same  dimensions  as  y{t)  -  output.  g()  and  /z()  are  the 
input  and  output  non-linearity  functions 
respectively.  B(q)  and  F(q)  are  regression 
polynomials. 

Figure  2 A  and  2B  represent  the  Hammerstein- 
Wiener  models  individually  and  their  combination 
respectively.  The  model  fit  values  are  computed 
using  Equation  (10)  as  follows; 


l—l y—y\ 

fit  =  ioo»^lll 

\\y-y  | 


(10) 


where,  y  is  the  estimated  output  by  the  model. 

The  linear  block  is  specified  using  the  terms  nb  -  the 
number  of  zeros  plus  one,  rij-  -  the  number  of  poles 


and  nk-  the  delay  from  the  input  to  the  output  in 

terms  of  the  number  of  samples.  The  commonly 
used  nonlinear  estimators  for  Hammerstein- Wiener 
model  are,  a)  Dead  Zone,  b)  Piecewise  Linear,  c) 
Saturation,  d)  Sigmoid  Network,  and  e)  Wavelet 
Network,  [15]. 

Genetic  Algorithm  (GA) 

Genetic  Algorithm  is  a  class  of  evolutionary 
methods  for  solving  both  constrained  and 


unconstrained  optimization  problems  that  are  based 
on  natural  selection. 


I  v  I  -  JST  y  Vil  1  m  ■ 

Figure  1:  Force  sensitive  Resistor  and  Thumb  Restrain 


(a) 


(b) 

Figure  2A:  a)  Hammerstein  model,  b)  Wiener  model 
Input  °utPut 


u(t) 


y(t) 


Figure  2B:  Hammerstein- Wiener  Model  Structure 

This  is  the  same  rule  that  governs  biological 
systems.  In  GA,  the  population  of  individual 
solutions  is  modified  repeatedly.  A  solution  is  given 
by  a  set  of  parameters  (genes)  and  packaged  as  a 
chromosome.  Each  step  or  iteration  the  GA  selects 
individuals  (chromosomes)  in  a  fashion  that  may 
include  some  randomness,  from  a  given  population, 
as  parents,  and  uses  them  to  produce  offspring  for 
the  next  generation.  Over  successive  iterations  or 
generations,  the  population  evolves  and  finally 
reaches  an  optimal  solution.  The  steps  that  make  up 
GA  are  as  follows: 

1)  Generate  a  random  population  of  4 p ’ 
chromosomes  -  these  chromosomes  carry 
information  of  the  population  and  are  confined  in 
the  feasible  solution  space.  2)  Evaluate  the  objective 
or  fitness  function  /  (p)  for  each  chromosome.  3) 
Create  a  new  population  or  offspring  from  the  initial 
population  by  using  certain  rules.  These  rules 
include  a)  Selection  -  selection  of  two  parent 
chromosomes  from  the  population  according  to  their 
fitness,  b)  Crossover  -  crossover  the  parents  to  form 
new  offspring,  if  no  crossover  is  performed  then  the 
offspring  is  an  exact  copy  of  the  parents,  c) 
Mutation  -  involves  the  changing  of  a  variable  in  a 
chromosome  or  some  other  change  in  the  original 
chromosome  as  defined  by  the  user,  d)  Acceptance 
Condition  -  if  offspring  satisfies  the  acceptance 
condition,  include  offspring  in  the  new  population  or 
else  discard.  4)  Use  the  offspring  as  the  parents  for 
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the  generation  of  a  new  population.  5)  Continue 
until  the  end  condition  is  satisfied.  Figure  4  presents 
a  graphical  interpretation  of  the  steps  in  GA.  In  this 
work,  GA  is  used  to  assist  in  finding  the  optimal 
values  of  the  filter  mask  for  the  filtration  of  the 
sEMG  signal  using  the  various  spatial  filters.  The 
cost  function  that  the  GA  tries  to  minimize,  is  the 
model  fit  obtained  from  system  identification  of  the 
sEMG-force  data  that  was  collected. 


Initial  Population 


Crossover 
Generation  1 


Mutated 
Generation  1 


Figure  4:  General  steps  in  a  binary  genetic  algorithm 


4  Results 

The  results  of  spatially  filtered  data  for  the  linear 
and  the  nonlinear  spatial  filters  were  first  compared 
based  on  the  “ Kurtosis ”  criteria,  [14].  ‘Kurtosis’  is  a 
measure  of  whether  the  data  sets  are  peaked  or  flat 
relative  to  a  normal  distribution.  That  is,  data  sets 
with  high  kurtosis  tend  to  have  a  distinct  peak  near 
the  mean,  decline  rather  rapidly,  and  have  heavy 
tails.  Data  sets  with  low  kurtosis  tend  to  have  a  flat 
top  near  the  mean  rather  than  a  sharp  peak.  A 
uniform  distribution  would  be  the  extreme  case.  The 
mathematical  expression  for  kurtosis  is  given  in 
Equation  1 1 . 


Kurt  = 


£[x4]-3(E[x2])2 


(ii) 


Figure  5  shows  the  plot  obtained  for  the  various 
filters  based  on  the  kurtosis  measure.  The  y-axis  is 
the  kurtosis  improvements  which  is  the  ratio  of  the 
output  to  the  input  kurtosis.  It  is  evident  from  the 
plot  that  the  NLT,  NLL,  NLTO  and  the  NLAFD 
spatial  filters  performed  very  well  as  compared  to 
the  linear  spatial  filters  under  investigation.  The  best 
kurtosis  improvements  were  obtained  for  experiment 
4  ~  44  for  the  nonlinear  spatial  filters.  The  x-axis  in 
Figure  5  shows  the  numbers  allotted  to  the  various 
spatial  filters  investigated.  Based  on  these  results  we 
focused  our  attention  to  the  linear  spatial  filter 
masks  for  experiment  3,  which  exhibits  poor 
kurtosis  values.  The  filters  were  compared  based  on 
the  model  fit  values  obtained  from  various 
Hammerstein- Wiener  models.  The  MatlabR  code  for 
the  Hammerstein- Wiener  model  is:  nlhw(ze,  [ nb  nj- 
nk],  ....,  ...).  The  modeling  was  carried  out  by 
varying  nb  -  the  number  of  zeros  plus  one,  Uj-  -  the 


number  of  poles  and  n b  -  the  delay  from  input  and 
output  in  terms  of  the  number  of  samples  for  the 
various  Hammerstein- Wiener  models. 


Types  of  Spatial  Filters 


Figure  5:  Kurtosis  plots  of  various  filters  for  exp3  -  y-axis 
numbers  are  for  the  various  filters  1-NLT,  2-NLL,  3 -NLTO,  4- 
NLAFD,  5-LSD,  6-TSD,  7-LDD,  8-TDD,  9-NDD,  10-IB2,  11- 
IR 

In  all,  42  models  with  variations  in  na  and  nb  were 
tested  while  the  value  of  nk  was  kept  as  1 .  The  total 
number  of  models  estimated  were  7  (filter  types)  x  4 
(time  windows)  x  42  models  per  time  window  = 
1,176  models.  The  time  windows  used  for 
estimation  and  validation  of  the  models  were  called 
‘ze  ’  and  ‘zv  ’  respectively,  ‘ze  ’  contained  8000 
sample  points  and  ‘zv  ’  contained  data  points  shifted 
by  2000  sample  points.  For  example,  if  ‘ ze ’  was  a 
time  window  between  2-6  seconds  i.e.  samples 
4000-12000,  then  ‘zv'  was  between  3-7  seconds  i.e. 
6000-14000  samples.  These  models  were  computed 
using  the  filter  masks  available  in  literature.  On 
examining  the  fit  values  closely  we  found  them  to 
have  large  variation  from  one  model  order  to 
another.  One  of  the  reasons  of  this  could  be  due  to 
the  poor  correlation  in  the  estimation  and  validation 
data  sets,  on  account  of  the  large  variations  that 
were  achieved  in  the  force.  This  paper  does  not  list 
all  these  models  tested  but  identifies  and  reports 
only  the  significant  results  of  the  analysis.  We  found 
that  the  filters  tested  performed  poorly  in  the  initial 
time  window  of  2-7  seconds  for  experiment  3.  Some 
of  the  models  for  this  time  window  were  selected 
and  then  recomputed  using  GA  to  optimize  the  filter 
mask  with  the  fitness  function  being  the  fit  value  of 
an  identified  model  achieved  for  a  given  model 
order.  Table  1  shows  an  example  of  the  fits  that 
were  obtained  on  varying  the  parameters  of  the 
Hammerstein- Wiener  model.  The  highlighted 
models  (and  a  few  other  models)  were  selected 
randomly  to  be  optimized  using  GA.  We  had  two 
scenarios  under  GA  -1)  GA  Constrained  and  2)  GA 
-  Unconstrained.  The  first  scenario  GA  constrained 
optimized  only  the  mask  entry  a22  (location  of  the 
sEMG  sensor  on  the  motor  unit).  The  other  entries 
of  the  filter  mask  were  then  computed  from  this 
optimized  value.  In  the  second  scenario,  GA 
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unconstrained  we  let  GA  optimize  all  the  entries  for 
various  masks.  The  GA  parameters  for  optimization 
were  as  follows:  number  of  iterations:  50;  initial 


population  size  Generation  0:  96;  population  size 
Generation  1:  48;  Number  of  Chromosomes  kept  for 


Model 

Number 

na 

nb 

nk 

LDD 

LSD 

TDD 

TSD 

NDD 

IB2 

IR 

ml 

2 

3 

1 

BlM 

■ 

39.53 

-9.685 

45.55 

33.79 

35.38 

m2 

2 

4 

1 

31.86 

17.4 

39.36 

43.1^ 

36.84 

16.88 

23.89 

m3 

2 

5 

1 

23.54 

35.94 

30.09 

2.247 

39.778 

p6.U| 

38.16 

m4 

2 

6 

1 

3.266 

Na.99 

41.28 

43.46 

10.83 

36.41 

36.7 

m5 

2 

7 

1 

31.58 

40.55 

44.65 

31.28 

2.593 

32.26 

36.34 

m6 

2 

8 

1 

37.84 

4.259 

21.89 

20.08 

8.036 

36.22 

36.57 

ml 

2 

9 

1 

31.42 

-0.1887 

35.78 

28.71 

MM 

35.54 

35.82 

m8 

3 

3 

1 

36.59 

40.04 

36.17 

12.33 

4.478 

am 

35.81 

m9 

3 

4 

1 

20.74 

40.72 

31.53 

36.57 

29.76 

37.43 

38.76 

mlO 

3 

5 

1 

39.87 

11.53 

19.831 

30.24 

39.25 

35.15 

37.9 

! 

! 

; 

! 

! 

; 

! 

! 

! 

! 

m40 

7 

7 

1 

34.91 

10.18 

40.79 

35.21 

35.24 

33.42 

37.92 

m41 

7 

8 

1 

-0.1127 

36.28 

39.68 

40.26 

20.27 

35.88 

24.32 

m42 

7 

9 

1 

36.55 

-0.412 

25.24 

31.31 

35.4 

34.26 

39.82 

Table  1:  Example  of  System  Identification  Results  Using  Filter  Mask  from  Literature  -  Highlighted  Models  Optimized 


Longitudinal  Single  Differential  (LSD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

41.39 

59.8055 

m4 

41.99 

41.5731 

68.4191 

m8 

40.04 

40.5924 

63.8523 

m9 

40.72 

56.5113 

Longitudinal  Double  Differential  (LDD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

31.06 

61.3475 

m6 

37.84 

33.724 

ml2 

35.54 

51.7942 

m31 

39.44 

60.6489 

Transverse  Single  Differential  (TSD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

m2 

43.19 

65.9038 

m4 

43.46 

58.2755 

mil 

44.24 

39.7867 

44.8431 

m3 1 

29.1 

47.6688 

Transverse  Double  Differential  (TDD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

39.53 

59.8055 

m5 

44.65 

68.4191 

mlO 

39.83 

63.8523 

ml  5 

43.01 

43.3466 

56.5113 

Normal  Double  Differential  (NDD) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

45.55 

57.9496 

m3 

39.87 

60.7374 

m7 

38.11 

60.5535 

mil 

39.01 

56.1193 

Inverse  Binomial  2  (IB2) 

Fit  % 

From 

Literature 

GA 

Constrained 

GA 

Unconstrained 

ml 

33.79 

58.2365 

m3 

36.17 

58.1653 

m4 

36.41 

55.5179 

m8 

41.09 

55.0537 

Table  2:  Results  of  Constrained  and  Unconstrained  GA  of  Highlighted  Models  from  Table  1 
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From  the  results  in  Table  2  we  can  see  that  the 
optimization  of  the  filter  mask  using  GA  worked  in 
almost  all  the  cases  chosen.  GA  without  constraints 
performed  significantly  better,  in  most  cases,  than 
the  filter  masks  reported  in  literature  and  also  the 
mask  which  we  computed  using  GA,  which  only 
optimized  the  entry  (a22)  i.e.  the  weight  associated 
with  the  sEMG  signal  at  the  motor  unit.  This 
restriction  on  GA  would  leave  the  filter  mask 
symmetrical.  But  looking  at  the  results  of  the  GA, 
we  can  conclude  that  the  filter  mask  need  not  always 
be  symmetrical  for  analysis  of  sEMG,  especially  for 
data  recorded  using  an  array.  Examples  of  the  filter 
mask  that  we  obtained  for  TDD,  model  m5  and 
NDD,  model  3  spatial  filters  are  given  below: 

"0  -41.5561  0" 


TDD  = 


0 


54.7329 


0 


0  -22  0 

0  -11.5790  0 


NDD  = 


-46.7773 


36.9276 


-11.8061 


0  -70.0491  0 


4  Conclusion 

The  linear  spatial  filter  masks  reported  in  literature 
had  poor  kurtosis  improvements  which  we 
optimized  using  GA,  these  were  compared  based  on 
the  model  fit  values  achieved  using  Hammerstein- 
Wiener  models.  The  fit  values  did  improve 
significantly  in  the  two  GA  scenarios  -  GA  with 
and  without  constraints.  The  GA  without  constraints 
performed  better  than  the  GA  with  constrains, 
which  brings  into  focus  the  possibility  that  the 
sEMG  signal  distribution  over  the  entire  grid  cannot 
be  assumed  to  be  symmetrically  distributed  and  that 
the  weights  associated  with  the  sEMG  signal  at 
various  locations  need  to  be  modified  depending  on 
probably  the  subject  and  also  based  on  the 
experimental  design.  This  is  in  contrast  to  the 
reported  filter  mask  in  the  literature,  which  are  all 
symmetric.  Almost  all  the  filter  masks  optimized 
resulted  in  a  significant  improvement  over  the 
masks  reported  in  literature. 
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ABSTRACT 

Traditional  electromyopgrahic  (EMG)  measurements  are 
based  on  single  sensor  information.  Due  to  the  arrangement  of 
skeletal  muscle  fibers  for  hand  motions,  cross  talk  is  an  inherent 
problem  when  inferring  motion/force  potentials  from  EMG  data. 
This  paper  studies  means  of  using  sensor  arrays  to  infer  better 
motion/force  potential  for  prosthetic  hands.  In  particular,  a 
surface  electromyographic  (sEMG)  sensor  array  is  used  to 
investigate  multiple  model fusion  techniques.  This  paper  provides 
a  comparison  between  three  statistical  model  selection  criteria. 
The  sEMG  signals  are  pre-processed  using  four  filters, 
Butterworth,  Chebyshev  type-II,  as  well  as  Bayesian  filters  such 
as  the  Exponential  and  Half-Gaussian  filter.  Output  Error  (OE) 
models  were  extracted from  sEMG  data  and  hand  force  data  and 
compared  using  a  Bayesian  based fusion  model.  The  four  different 
filters  effect  were  quantified  based  on  the  OE  models  performance 
in  matching  the  actual  measured  data.  The  comparison  indicates 
a  preference  for  using  the  sensor  fusion  technique  with 
preprocessed  EMG  data  using  the  Half-Gaussian  Bayesian  filter 
and  the  Kullback  Information  Criterion  (KIC). 

NOMENCLATURE 

B(q),  F(q)  Polynomials  for  parametric  model. 

P(Mi  | z)  Probability  of  model  Mt  given  data  z 

R  Residual  norm. 

Y  Force  data  (measured). 


Y  Predicted  force  (system  output). 

Y  Mean  value  of  force  data  (measured). 

e(t)  Error  sequence. 

n  Number  of  data  points. 

Pi  Order  of  the  ith  model  ( M( ). 

u(t)  Measured  sEMG  data  (system  input). 

0  EMG  and  Force  data. 

6  Parameter  vector. 

if 7  Digamma  function. 

INTRODUCTION 

According  to  the  information  provided  by  the  National 
Center  for  Health  Statistics,  every  year  there  will  be  50,000  new 
amputations  in  the  USA.  In  recent  years,  much  progress  has  been 
made  in  developing  prosthetics  for  individuals  who  have 
sustained  hand  amputation.  A  traditional  approach  to  control  hand 
prosthetics  is  based  on  surface  myographic  (sEMG)  signals, 
gathered  through  electrodes  placed  on  the  skin’s  surface,  and 
hence  do  not  require  the  implementation  of  needle  sensors,  or  the 
direct  wiring  of  nerves  to  external  devices.  This  paper  details  the 
data  processing  of  sEMG  recordings  for  the  purpose  of 
controlling  smart  hand  prosthetics.  Surface  electromyography  is 
an  area  of  research  that  began  some  60  years  ago.  Over  time, 
many  methods  for  filtering  of  EMG  signals  were  proposed  for 
different  applications.  Recently,  a  novel  nonlinear  recursive  filter 
based  on  Bayesian  estimation  [1]  was  proposed  in  order  to 
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overcome  the  limitation  of  the  EMG  signal  usage  in  the  fields  of 
neural  control,  prosthetics  and  robotics.  This  filter  allows  the 
signal  to  maintain  the  same  amplitude  during  the  constant  drive 
and  allows  the  recording  of  rapid  changes  in  the  signal  instead  of 
smoothing  the  signal  in  the  time  domain. 

Since  sEMG  sensors  are  prone  to  pick  up  cross  talk  from 
adjacent  muscle  groups,  filtering  of  such  data  is  an  important  step 
in  order  to  infer  the  intended  finger  motion.  Recently  a  multi¬ 
channel  electrode  approach  was  introduced  to  infer  the  EMG 
signal,  [2].  This  system  consisted  of  96  different  EMG  surface 
electrodes,  where  one  line  (16  channels)  was  selected  based  on 
the  noise  contents.  The  data  from  this  line  went  through  a 
canonical  discriminate  analysis  to  select  the  three  first 
eigenvectors,  which  are  used  to  classify  the  EMG  signal.  This 
approach  allows  tracking  of  12  movements  of  the  hand  including 
four  finger  flexion  movements  with  an  80%  success  rate.  In  this 
paper,  we  propose  to  use  as  well  a  sensor  array.  Though  we  are 
not  interested  in  the  classification  of  motion  as  done  in  the  work 
by  Natata  et.  al  [2],  rather  we  aim  to  extract  dynamical  models 
that  describe  the  sEMG  -  finger  force  relationship.  Also,  we 
propose  to  use  all  sensor  data  within  the  array  and  apply  a  sensor 
fusion  technique  in  order  to  maximize  the  model  accuracy. 
Multiple  model  fusion,  as  detailed  in  [3],  is  based  on  assigning  a 
particular  probability  to  each  individual  model,  which  is  extracted 
using  System  Identification  (SI)  from  a  single  sEMG  sensor.  The 
probability  is  based  on  the  SI  model  fit  value,  given  by 

[i-|z-f|/|z-F|]xioo  . 

Considering  recorded  sEMG  signals  as  the  input  and  the 
resulting  force  signal  generated  by  the  finger  movements  of  the 
hand  as  the  output,  the  raw  signal  will  need  to  be  processed.  In 
this  work  we  will  investigate  the  effect  of  different  filters  such  as 
Butterworth,  Chebyshev,  and  nonlinear  recursive  filters  that  are 
based  on  Bayesian  estimation  using  measurement  models  such  as 
Exponential  and  Half  -Gaussian,  for  details  on  these  filters  see 
[1].  In  addition,  we  will  be  investigating  the  usage  of  different 
model  selection  criterion  for  the  use  of  multiple  model  fusion. 

THEORETICAL  BACKGROUND 

The  low-pass  Chebyshev  type  II  filter  in  this  work  is 
constructed  with  a  cutoff  frequency  of  550Hz,  while  the  low  pass 
Butterworth  filter  is  of  order  12.  The  Bayesian  filter  has  the 
capability  of  recording  the  rapid  changes  in  the  sEMG  signal 
instead  of  just  smoothing  it.  It  was  shown  in  [1]  that  the 
processed  data  will  have  a  high  signal-to-noise  ratio  and  a 
relatively  low  error  as  compared  to  low-pass  5Hz,  low-pass  1Hz, 
or  low-pass  0. 1Hz  linear  filters.  A  Bayesian  filtering  method  may 
potentially  provide  a  good  EMG  signal  for  classification 
purposes.  The  filter  has  not  been  investigated  in  terms  of  the  use 
for  system  identification,  where  the  output  force  generated  by  the 
hand  is  modeled  based  on  the  measured  sEMG  data. 

System  Identification:  The  system  identification  is  accomplished 
by  utilizing  Output  Error  (OE)  models.  The  filtered  and  rectified 


EMG  data  is  used  to  extract  OE  models,  for  more  details  see  [4]. 
The  corresponding  equation  for  the  OE  model  is  given  by 


B(g) 

M.  =  7(0  =  -I-  u{t  -  nk )  +  e(t)  . 
*  F(q) 


(1) 


Model  Fusion:  Three  different  model  selection  criteria  are  used 
in  this  paper,  i.e.  the  Akaike  information  criterion  (AIC), 
Kullback  information  criterion  (KIC)  and  the  Bayesian 
information  criterion  (BIC),  for  more  details  on  the  specifics  of 
these  information  criterions  see  [3].  AIC  is  relatively  simple  to 
apply  because  it  requires  only  the  maximum  likelihood  to  be 
achieved  within  a  given  model.  AIC  is  an  asymptotically  unbiased 
estimator  of  the  Kullback’s  directed  divergence  between  the  true 
model  and  the  estimated  model.  The  dissimilarity  between  the 
two  statistical  models  is  called  relative  entropy  or  the  I- 
divergence.  AIC  becomes  a  negatively  biased  estimator  of  the 
model  when  the  dimension  of  the  candidate  model  increases  - 
when  compared  to  the  sample  size.  In  order  to  overcome  this 
shortcoming,  biased  corrected  version  of  AIC  was  proposed  in 
[5].  Akaike’s  asymptotic  bias  corrected  form  is  given  by 

AIC  =  -2  In  Rj  +2Pi.  (2) 

The  biased  corrected  version  by  Hurvich  is  stated  as 
n  (p-+l)n 

AIC(Pi )  =  -  log  R. ,  +  ’ -  .  (3) 

2  n-Pi-  2 

Kullback’s  information  criterion  is  an  asymmetric  measure. 
The  measure  of  the  models  dissimilarity  can  be  obtained  by  the 
sum  of  two  directed  divergences,  known  as  Kullback’s  symmetric 
or  J-divergence  [5]. 

KIC{Pi )  =  -  log  Ri  +  — * - n¥( - '-)  +  g(n),  (4) 

2  n-Pi-  2  2 

where  g(n )  =  nx  log  (n  /  2) .  Bayesian  inference  is  a  preferred  tool 


as  it  is  precisely  the  quantity  that  updates  the  probability  of  the 
prior  model  to  the  posterior  model  probability.  BIC  was 
introduced  by  Schwarz  in  1978,  [6]  and  comes  from 
approximating  evidence  ratios  of  models.  This  ratio  is  known  as 
the  Bayes  factor,  [6]. 


n  Pi  + 1 

BIC(Pj )  =  -  log  Rt  +  - log  n  ■ 


(5) 


Fusion  Algorithm 

The  following  is  the  fusion  technique  based  on  [3]  applied  to 
sEMG  -  force  identification  models. 

1 . )  Identify  OE  models  Mh  M2,  . . .,  Mk  using  sEMG  data  ( u )  as 

input  and  force  data  ( 7 )  as  output,  for  k  number  of  sensors 
collecting  data  simultaneously. 

2. )  Compute  the  residual  square  norm 

r.  =||  y  -  <D -<9-  ||2  =  ||y-y|| ,  where  0.  =  {df  O.}"1  Of  7 , 
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3. )  Calculate  the  model  criteria  coefficient  using  Eqs.  (3-5). 

4. )  Compute  the  model  probability 

e~l> 

p(Mj  |  Z)  =  — —j—  ,  where  /  is  the  respective  model 

I  e  j 

j= 1 

selection  criteria  coefficient,  i.e.  AIC(pi) ,  BIC(pj) ,  or 
KIC(Pl)  . 

A  k 

5. )  Compute  the  fused  model  output  7  =  Z  p{M-  \  Z)Y- . 

J  i= i  *  * 

6. )  Compute  overall  OE  model  from  Yj-  and  Force  data. 

Experimental  set  up:  In  order  to  identify  appropriate  EMG 
electrode  attachment  points,  a  wet  probe  point  stimulator  was 
used  at  the  forearm  superficial  musculature  (flexor  digitorum 
superficialis)  (RICH-MAR,  HV  1000)  for  collecting  flexion  of 
the  ring  finger  (fourth  digit).  Surface  EMG  electrodes  (model 
DE2.1;  Delsys)  were  placed  directly  on  the  marked  areas  to 
record  EMG  actively.  Force  was  measured  using  Interlink 
Electronics  FSR  0.5”  circle  sensors.  The  sEMG  and  the  force  data 
were  measured  simultaneously  at  a  sampling  of  2000Hz  for 
duration  of  10  seconds. 


TABLE  1:  MODEL  FUSION  RESULTS 


Criterion 

Models 

Butter- 

worth 

Cheby- 
shev  II 

Bayes- 

Expon. 

Bayes- 
Half  G. 

Mi 

54.3 

54.62 

55.24 

41.37 

m2 

61.24 

61.15 

55.24 

54.27 

BIC 

m3 

53.5 

63.0 

49.79 

64.73 

M(Yf) 

68.22 

76.02 

73.47 

80.52 

Mx 

54.3 

54.62 

55.24 

41.37 

m2 

61.24 

61.15 

55.24 

54.27 

AIC 

m3 

53.5 

63.0 

49.79 

64.73 

M{if) 

67.88 

75.65 

81.35 

92.89 

Ml 

54.3 

54.62 

55.24 

41.37 

m2 

61.24 

61.15 

55.24 

54.27 

KIC 

m3 

53.5 

63.0 

49.79 

64.73 

MOf) 

67.44 

71.02 

87.78 

93.74 

DISCUSSION 

Table  1  details  the  obtained  results  using  the  different 
selection  criteria  for  model  fusion  and  filter  options.  The  fit 
values  describe  how  well  the  particular  model  (Mi),  selected  filter, 
and  selection  criteria  predict  the  intended  finger  force.  The 
numbers  are  given  as  percentages,  where  100  represents  a  perfect 
fit  and  0  no  fit.  It  is  evident  that  the  general  approach  of 
combining  data  in  order  to  produce  an  overall  model  ( M(Y ) ) 
improves  the  resulting  force  prediction,  regardless  of  selection 
criteria  or  filtering  method.  From  the  results,  it  appears,  regardless 
of  the  model  selection  criteria  that  the  Half-Gaussian  filter  is 
performing  better  than  the  three  other  filters  investigated  with 
respect  to  model  fusion.  In  addition,  the  KIC  in  connection  with 
the  Half-Gaussian  filter  yields  the  best  fit  of  all  the  fused  models. 
This  is  true  also  if  the  other  Bayesian  filter  is  used.  The  AIC  is 
susceptible  to  over  fitting  and  works  well  if  the  true  model  order 
of  the  system  is  used.  Analogously,  the  BIC  is  receptive  to  the 
same  over  fitting  issue  as  the  AIC  and  is  sensitive  to  increases  in 
additional  parameters.  The  used  model  fusion  algorithm  works 
well  for  sEMG-force  relationships.  The  investigation  presented 
here  relay  on  one  particular  system  identification  routine.  Its 
impact  or  its  optimization  in  terms  of  model  representation  and/or 
estimation  and  its  application  to  hand  motion  models  without 
force  will  be  investigated  in  future.  The  presented  work  is  suitable 
for  the  use  of  array  surface  EMG  sensors. 
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Abstract —  Extracting  or  estimating  skeletal  hand/finger 
forces  using  surface  electro  myographic  (sEMG)  signals  poses 
many  challenges  due  to  cross-talk,  noise,  and  a  temporal  and 
spatially  modulated  signal  characteristics.  Normal  sEMG 
measurements  are  based  on  single  sensor  data.  In  this  paper, 
array  sensors  are  used  along  with  a  proposed  sensor  fusion 
scheme  that  result  in  a  simple  Multi-Input-Single-Output 
(MISO)  transfer  function.  Experimental  data  is  used  along  with 
system  identification  to  find  this  MISO  system.  A  Genetic 
Algorithm  (GA)  approach  is  employed  to  optimize  the 
characteristics  of  the  MISO  system.  The  proposed  fusion-based 
approach  is  tested  experimentally  and  indicates  improvement 
in  finger/hand  force  estimation. 

I.  Introduction 

The  number  of  people  with  missing  limbs  due  to  combat 
and  non-combat  operations  in  the  United  States  is  over  1.2 
million  according  to  [1].  The  recent  wars  in  Afghanistan  and 
Iraq  substantially  increased  the  number  of  amputees.  To  date 
there  are  no  prosthetic  devices  available  at  an  affordable  cost 
and  with  the  full  functionality  of  a  human  hand.  The  need 
for  such  a  device  in  these  days  is  becoming  more  important. 
There  are  multiple  types  of  hand  prostheses.  One  of  these 
types  is  based  on  measuring  sEMG  signals  to  initiate  the 
actuation  of  the  robotic  hand.  In  this  paper,  surface  based 
EMG  sensors  are  used  in  order  to  avoid  invasive  methods  of 
measuring  EMG  signals  [2].  The  major  problem  in  capturing 
the  sEMG  signal  is  cross-talk  from  different  motor  units.  As 
a  muscle  contracts  several  motor  units  fire  simultaneously 
and  hence  make  up  the  EMG  signal  [3]. 

Chandrasekhar  Potluri  is  with  Measurement  and  Control  Engineering 
Research  Center  (MCERC),  Idaho  State  University,  Pocatello,  Idaho  83201 
USA  (e-mail:  potlchan@isu.edu). 

Parmod  Kumar  is  with  MCERC,  Idaho  State  University,  Pocatello,  Idaho 
83201  USA  (e-mail:  kumaparm@isu.edu). 

Madhavi  Anugolu  is  with  MCERC,  Idaho  State  University,  Pocatello, 
Idaho  83201  USA  (e-mail:  anugmadh@isu.edu). 

Alex  Urfer  is  with  Department  of  Physical  and  Occupational  Therapy, 
Idaho  State  University,  Pocatello,  Idaho  83201  USA  (e-mail: 
urfealex@isu.edu). 

Steve  Chiu  is  with  Department  of  Electrical  Engineering  and  Computer 
Science,  MCERC,  Idaho  State  University,  Pocatello,  Idaho  83201  USA  (e- 
mail :  chiustev@isu .  edu) . 

D.  Subbaram  Naidu  is  with  Department  of  Electrical  Engineering  and 
Computer  Science,  MCERC,  Idaho  State  University,  Pocatello,  Idaho 
83201  USA  (e-mail:  naiduds@isu.edu). 

Marco  P.  Schoen  is  with  Department  of  Mechanical  Engineering, 
MCERC,  Idaho  State  University,  Pocatello,  Idaho  83201  USA  (e-mail: 
schomarc@isu.edu). 


The  number  of  motor  units  in  individual  muscles  varies 
from  person  to  person  and  muscle  fatigue  begins  at  the  onset 
of  contraction  and  is  ongoing  throughout  (i.e.)  time  to 
contraction  failure  or  reduction  in  force  generating  capacity 
of  the  muscle.  The  EMG  signal  passes  through  several  layers 
of  tissues  before  reaching  the  skin  surface  [3].  This 
contributes  to  noise  and  interference  in  signal  acquisition. 
The  randomness  of  the  EMG  signal  is  an  added  difficulty  in 
studying  it.  All  of  these  factors  make  it  rather  complex  to 
discriminate  the  content  of  the  EMG  signal  against  noise  and 
interference.  Identification  of  the  content  of  the  EMG  signal 
should  focus  on  the  amplitude,  frequency  and  amplitude- 
frequency  content  using  distinct  methods.  Since  the  signal  is 
amplitude  modulated  and  time  dependent,  it  is  therefore 
spatially  and  temporally  frequency  encoded  [4]. 

An  approach  to  distinguish  the  onset  of  muscle 
contraction  was  introduced  by  Merlo  et  al.  [5].  In  their  study, 
the  sEMG  sensors  collect  the  action  potential  of  individual 
motor  units  which  are  in  turn  identified  by  using  a 
continuous  wavelet  transform.  Accuracy  of  determining  the 
intended  force  from  the  EMG  signal  is  improved  by  using  an 
array  sensor.  Array  sensors  use  multi-channel  EMG 
electrodes  embedded  into  a  small  surface  area  in  order  to 
capture  the  electrical  muscle  activity  from  a  set  of  muscles 
fibers  at  the  skin  level.  The  EMG  sensor  array  approach 
produces  a  micro  level  and  a  macro  level  sensor  network  by 
utilizing  several  array  sensors  at  different  skin  locations  of 
the  forearm.  An  array  of  micro  sensors  with  the  objective  to 
extract  signals  from  one  motor  unit  corresponds  to  the  micro 
level.  The  main  objective  of  this  arrangement  is  to  use 
redundant  data  to  infer  the  hand  motion  by  including  cross¬ 
talk.  Also,  a  multi-channel  methodology  is  proposed  where 
elements  of  the  sensor  array  is  discarded  based  on  the 
quality  of  the  signal  [6].  A  simple  sensor  fusion  scheme  to 
improve  the  estimated  hand/fmger  force  from  measured 
sEMG  data  based  on  Bayesian  approaches  was  given  [7]. 

In  the  following,  Genetic  Algorithm  (GA)  is  used  to 
accomplish  the  sensor  fusion  algorithm  [8].  Briefly,  GA’s 
are  evolutionary  algorithms  that  simulate  Darwin’s  survival 
of  the  fittest  principle.  The  initial  population  of  candidate 
solutions  is  randomly  generated  and  represented  as 
chromosomes  in  the  form  of  genes.  These  chromosomes  are 
evaluated  based  on  an  objective  function  and  ranked  in  terms 
of  its  fitness.  A  subset  of  the  next  generation  of  candidate 
solutions  is  selected  based  on  their  performance  with  the 
objective  function.  The  remaining  set  of  the  new  generation 
is  generated  by  a  mating  process,  where  the  best  performing 
candidate  solution  comprise  the  subset  of  the  parents.  In 
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addition  to  the  mating  process,  a  mutation  rate  is  also 
embedded  in  the  generation  of  the  new  population.  The 
mutation  rate  enables  the  search  for  the  optimum  solution  to 
overcome  local  minimums  and  locate  the  global  minimum. 
This  process  of  selection,  mating,  and  mutation  is  repeated  a 
number  of  times  until  the  best  performing  candidate  solution 
converges  to  some  stationary  value.  GA’s  have  the 
capability  of  overcoming  local  minima,  some  additional 
advantages  of  a  genetic  algorithm  are  the  ease  with  which 
large  numbers  of  parameters  can  be  handled,  the  fact  that 
they  do  not  require  the  traditional  approach  of  taking 
derivatives,  the  fact  that  they  result  in  a  set  of  optimum 
candidate  solutions  rather  than  a  single  candidate  solution, 
and  that  they  work  well  with  experimental  data  as  well  as 
simulated  data. 

The  approach  presented  in  this  paper  is  based  on 
extracting  useful  information  from  all  sensors  in  the  array. 
To  infer  the  intended  motion  from  the  sensor  array,  we 
propose  to  use  an  optimized  non-linear  filter.  Optimization 
techniques  are  used  as  filters  to  fuse  the  EMG  data  from 
different  sensors.  The  fusion  process  is  done  in  the 
frequency  domain. 

The  paper  is  structured  as  follows.  The  experimental  set 
up  is  given  in  Section  II  and  Section  III  provides  background 
on  sensor  fusion.  Section  IV  details  the  results  and 
discussion  and  in  the  last  section  some  conclusions  are 
provided. 

II.  Experimental  set-up 

Experiments  were  carried  out  on  a  healthy  male  subject  in 
order  to  collect  EMG  data  from  all  the  sensors 
corresponding  to  each  individual  motor  unit.  The 
mcheppuotor  points  of  the  subject  and  the  appropriated 
EMG  electrode  attachment  points  were  identified  by  using  a 
wet  probe  point  muscle  stimulator  (Rich-Mar  Corporation, 
model  number  HV  1 100.).  For  capturing  the  EMG  data  from 
the  skin  surface,  a  Delsys,  Bagnoli-16  channel  EMG,  DS- 
160,  S/N-l  116  system  is  used.  This  capturing  system  has  the 
capability  of  internally  amplifying  and  reducing  the  acquired 
noise.  Pronged  DE  2.1  differential  surface  electrodes  are 
used.  For  recording  EMG  at  the  forearm  superficial 
musculature  (flexor  digitorum  superficialis)  electrodes  are 
placed  to  extract  EMG  signals.  The  force  data  is  measured 
by  a  force  sensitive  resistor  (FSR).  The  data  acquisition  is 
set  to  a  sampling  rate  of  2000  samples  per  second  for  force 
and  EMG  signal  data. 

III.  Sesnor  Fusion  Background 

The  sensor  fusion  process  deals  with  association, 
correlation  and  estimation  of  data  from  multiple  sensors.  A 
data  fusion  technique,  by  combining  the  data  from  different 
sensors,  enables  one  to  achieve  more  specific  inferences 
about  the  measured  data.  The  sEMG  sensors  pick  up  the 
cross-talk  from  muscle  fibers  adjacent  to  motor  units.  In  [7], 
sensor  fusion  was  done  in  the  time  domain,  by  taking  sEMG 
as  input  and  force  signal  as  the  output.  They  used 


combinations  of  different  filters  (Butterworth,  Chebyshev, 
Exponential  and  Half-Gaussian  filters)  and  different 
information  criteria  based  on  Akaike  (AIC),  Bayesian  (BIC), 
and  Kull-back  (KIC)  and  concluded  that  the  KIC  criterion 
with  Half  Gaussian  filtering  gives  the  best  EMG-Force 
model  fit. 

In  the  present  case,  sensor  fusion  is  done  in  the  frequency 
domain  for  the  sEMG  data.  Figure  1  depicts  the  flow 
diagram  of  the  proposed  fusion  technique  using  a  simple 
elitism  based  GA.  The  data  from  the  three  sensors  around 
the  individual  motor  unit  are  fused  by  first  rectifying  the 
data  and  then  filtering  it  using  a  Half-Gaussian  filter.  The 
sEMG  data  ulfu2,  and  u3  from  the  three  sensors  and  their 
corresponding  force  signal  is  used  to  identify  a  dynamical 
relationship  between  the  sEMG  signals  and  the  force  signals 
using  System  Identification  (SI).  In  this  proposed  fusion 
algorithm,  SI  is  achieved  by  utilizing  an  Output  Error  (OE) 
model  first  for  each  individual  data  set  [9].  The  OE  model 
structure  is  given  as  follows: 

y (0  =  nfc)  +  e(0>  (i) 

where  B ,  and  F  are  the  polynomials,  q  is  shift  operator,  e(t) 
is  output  error,  y(t)  is  system  output,  u  is  input,  nk  is  the 
system  delay  and  t  is  time  index. 

The  corresponding  continuous-time  model  is  given  by  the 
transfer  function. 

r(  \  £00  fcrtfts(nk~1)frnb-is(n*)~2)~l  Hfoi  n. 

K  }  F(s)  Snf+fnfSnf~  1+-.+A  •  W 

Similar  to  the  discrete -time  case  the  orders  of  the 
numerator  and  denominator  are  determined  by  nb  and  nf. 
For  multi-input  systems,  nb  and  nf  are  row  vectors.  b,f  are 
the  coefficients  of  the  numerator  and  denominator 
polynomials  respectively. 

Based  on  the  poles  of  the  three  individual  identified  OE 
models,  corresponding  to  each  sensor,  a  MISO  transfer 
function  (//)  is  constructed.  While  the  denominators  of  the 
respective  individual  transfer  functions  (corresponding  to 
each  OE  model)  are  transferred  over  to  the  new  MISO 
transfer  function,  the  corresponding  zeros  are  found  through 
the  use  of  a  GA.  Generally  GA’s  can  find  global  optimum 
points  if  elitism  is  used  and  sufficient  number  of  generations 
are  allowed  in  the  algorithm.  This  optimization  algorithm  is 
rather  computationally  expensive,  but  since  there  was  not  a 
computational  time  requirement,  one  is  free  to  use  GA  rather 
than  other  intelligent  based  algorithms.  Chromosomes  are 
constructed  by  designating  each  zero  of  a  numerator  as  a 
gene.  Since  a  discrete  time  model  is  utilized,  the  search  area 
is  limited  to  the  unit  circle  (and  the  resulting  MISO  model  is 
decreased  to  be  minimum  phase).  The  number  of  potential 
zeros  was  set  to  the  order  of  the  corresponding  denominator 
because  the  number  of  zeros  is  at  most  the  number  of  poles 
(for  a  causal  system,  can’t  predict  the  future).  The  objective 
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function  was  set  as  the  error  squared  of  the  resulting  MISO  validation  data  sets.  The  output  of  the  fused  model  exhibits 

system  (//)  (see  appendix)  and  the  recorded  force  signal.  an  82%  fit  with  the  actual  force  signal. 

Here  H  is  given  as, 
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where  Z’s  and  P’s  are  the  zeros  and  poles  respectively  of 
individual  transfer  function  and  n  is  the  order  of  the  system. 


ux 

u2 

u3 


Figure  1:  Flow  Diagram. 


IV.  Results  And  Discussion 


The  actual  force  signal  from  the  FSR  sensor  is  filtered 
using  a  Chebyshev  Type  II  high  pass  filter  followed  by  a  low 
pass  filter.  The  EMG  signal  is  rectified  and  filtered  using  a 
Half-Gaussian  filter  to  minimize  the  noise  content.  The  filter 
parameters  for  the  Half-Gaussian  filter  are  selected  in  such  a 
way  that  the  effective  EMG  signal  frequency  content  is  no 
more  than  500  Hz,  according  to  the  ISEK  standards  [10]. 
Figure  2  shows  the  results  of  the  fusion  based  force  Y 
plotted  against  actual  measured  force.  The  predicted  force 
matches  the  actual  force  characteristics  well.  Figure  3  shows 
the  estimated  force  output,  for  the  individual  OE  models 
which  were  used  to  construct  the  fusion  model  and  the  actual 
measured  force.  It  is  apparent  that  the  individual  force 
estimates  from  each  OE  model  do  not  track  the  actual  force 
as  well  as  the  estimated  force  from  the  fused  model  H.  In 
particular,  the  magnitudes  are  substantially  different.  While 
not  the  most  optimal  identification  was  determined,  the 
improvement  indicated  by  the  fused  model  output  compared 
to  the  individual  OE  outputs  is  significant.  The  output  of  the 
fused  model  fits  with  peaks  of  the  original  force  data. 
Several  validation  tests  for  the  fused  model  are  done  by 
feeding  new  data  sets  to  the  MISO  transfer  function  (//). 

Figure  4  and  5  shows  the  plot  between  the  actual  force 
data  and  the  predicted  force  data  for  two  representative 
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Figure  2  :  Final  Fused  Force  Vs.  Actual  Force. 
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Figure  3:  Estimated  Force  Output  for  Individual  OE  Model 
and  the  Actual  Measured  Force. 


Figure  4:  Validation  Plot  for  Data  Set-I. 
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— Actual  Force  for  New  Data  Set 


— Fusion  Based  Force  Output  for  New  Data  Set 
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Figure  5:  Validation  Plot  for  Data  Set-II. 


Figure  6:  Comparison  between  Actual,  Fusion  Based  and 
Single  OE  Model  Output  Force. 

In  order  to  visualize  the  improvement  achieved  with 
regard  to  the  form  of  the  predicted  force,  Figure  6  is 
generated  to  depict  the  actual  force  measured  compared  to 
the  output  of  the  fused  model  (predicted  force)  and  the 
predicted  force  from  one  of  the  OE  models  used  to  construct 
the  fused  model.  The  amplitudes  were  modulated  in  order  to 
show  the  different  shapes  of  the  wave  forms.  It  is  apparent 
that  the  fused  model  output  matches  the  measured  finger 
force  better  than  an  individual  OE  force  estimate.  While 
there  are  some  additional  higher  frequency  terms  in  the 
estimated  finger  force,  the  general  trend  and  lower 
frequencies  are  well  matched.  The  higher  frequency 
elements  are  necessary  parts  of  the  MISO  model  since  finger 
forces  can  change  rather  quickly,  as  indicated  in  Figure  2. 

V.  Conclusion 

Although  better  individual  models  can  be  inferred  using 
SI  under  perfect  conditions,  the  proposed  fusion  algorithm 
improves  the  predicted  force  estimate  consistency.  The 
fusion  algorithm  results  in  a  single  MISO  transfer  function. 
Cross  talk  can  be  reduced  by  using  filtering  and  in  this  work 
a  Half-Gaussian  filter  is  used.  This  also  enables  the 
smoothing  of  the  fused  signal.  Several  validation  data  sets 
were  used  to  test  the  proposed  fusion  algorithm.  Compared 
to  single  OE  model,  the  proposed  fusion-based  force  model 


is  in  good  agreement  with  that  of  the  actual  force.  Future 
work  will  address  the  optimization  of  the  SI  and  the 
associated  experimental  conditions  in  order  to  arrive  at 
better  individual  OE  models.  Also  the  optimization  of  the 
poles  and  the  number  of  zeros  will  be  undertaken  in  order  to 
obtain  a  minimum  realization  of  the  MISO  system.  In 
addition,  the  MISO  system  is  an  ideal  structure  to 
incorporate  dynamics  related  to  muscle  fatigue,  which  will 
be  addressed  in  future  work. 

Appendix 

The  resulting  MISO  transfer  function  H  is  constructed  as, 
From  ux  to  output, 

S8  — 3.843s7  +  7.729s6— 10 .78s5 +10.6S4— 7.417s* 1 2 3  +  3.603s2 5 6 7 8 9 10  — 0.9795S+0.1 192 
S8— 4.028s7  +  6.325s6— 4.121s5  — 1.545s4+5.87s3— 5.433s2+2.28s— 0.3496 

From  u2  to  output, 

S8-  4.339s7+9.005s6—  12.42s5  +  12.22s4-  8.117s3  +  3.427s2-  0.9134  s  +  0.1424 
S8  -  4.028  S7  +  6.325  S6  -  4.121  s5  -  1.545  S4  +  5.87  s3  -  5.433  S2  +  2.28  S  -  0.3496 

From  u3  to  output, 

s8  -  3.522  s7  +  6.655  s6  -  8.864  s5  +  8.183  s4  -  5.365  s3  +  2.423  s2  -  0.557  s  +  0.09585 

S8  -  4.028  S7  +  6.325  S6  -  4.121s5  -  1.545  S4  +  5.87  s3  -  5.433  S2  +  2.28  S  -  0.3496 
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Abstract —  Recently,  we  focused  our  investigations  on  the 
dynamic  modeling  of  the  skeletal  muscle  of  the  human  upper 
arm  using  surface  electromyographic  (sEMG)  signals.  The 
objective  of  these  investigations  is  to  develop  models  that 
dynamically  compute  the  skeletal  muscle  force  and  finger 
angles  for  prosthetic  hand  control  using  sEMG  as  input.  Since 
sEMG  is  temporal  and  spatially  distributed  and  is  influenced 
by  various  factors,  muscle  fatigue  and  its  related  sEMG 
becomes  of  importance.  This  study  is  an  effort  to  spectrally 
analyze  the  sEMG  signal  during  progression  of  muscle  fatigue. 
The  sEMG  is  captured  using  an  array  of  nine  sensors  from  the 
arms  of  healthy  subjects  during  muscle  fatiguing  experiments 
for  dynamic  and  static  force  levels.  The  sEMG  signal  is  first 
filtered  with  a  nonlinear  Teager-Kaiser  Energy  (TKE) 
operator-based  nonlinear  spatial  filter  and  segmented  in  five 
parts  with  75%  overlap  between  adjacent  segments.  The 
analysis  is  done  using  different  classical  (fast  Fourier 
transform,  Welch’s  averaged  modified  periodogram),  model- 
based  (Yule-Walker,  Burg,  Covariance  and  Modified 
Covariance  autoregressive  (AR)  method),  and  eigenvector 
methods  (Multiple  Signal  Classification  (MUSIC)  and 
eigenvector  spectral  estimation  method)  in  frequency  domain. 
Results  show  that  the  classical  and  eigenvector  based  methods 
are  more  sensitive  than  the  model-based  methods  to  fatigue 
related  changes  in  sEMG  signals. 

I.  Introduction 

THIS  work  focuses  on  the  analysis  of  sEMG  signals 
and  investigates  the  effects  of  fatigue  in  the  skeletal 
muscles.  The  sEMG  signal  is  an  electric  voltage  signal 
ranging  between  -5  to  +5  mV  and  is  generated  by  skeletal 
muscles.  The  motor  units  operate  as  a  consequence  of  the 
central  control  strategies,  signal  transmission  along  nerve 
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fibers  and  across  neuromuscular  junctions,  electrical 
activation  of  the  muscle  fibers  organized  in  elementary 
motors  and,  through  a  chain  of  complex  biochemical  events. 
The  productions  of  the  forces  acting  on  the  tendons  of  the 
agonist  and/or  antagonist  muscles  predict  bone  movement 
[1].  This  mechanism  also  involves  a  number  of  feedback 
circuits  relaying  back  to  the  spinal  cord  and  the  brain 
information  concerning  the  length  and  velocity  of  shortening 
of  the  muscles  and  the  forces  acting  on  the  tendons  [1].  The 
EMG  signal  gives  information  about  the  motor  points  and 
their  controller  i.e.  the  central  nervous  system  [2,  3].  The 
central  nervous  system  activates  and  controls  the  EMG 
signals  which  depend  on  the  flow  of  specific  ions  including 
sodium  ( Na+ ),  potassium  ( K+ )  and  calcium  ( Ca++ ) 
resulting  in  the  action  potentials  in  nerves  and  their 
respective  skeletal  muscle  fibers  from  which  EMG  signals 
are  derived.  Research  efforts  in  the  last  few  decades  have 
focused  on  the  prosthetic  design  where  sEMG  is  a  control 
input  to  activate  the  prosthesis.  The  sEMG  is  a  dynamic 
signal  with  continual  change  in  its  pattern  and  strength  and 
this  becomes  more  complex  with  the  fatigue  induced  in 
skeletal  muscles. 

Failure  to  maintain  the  required  force  level  is  termed  as 
muscle  fatigue,  which  is  a  complex  phenomenon  [4]. 
Reasons  for  fatigue  can  be  the  peripheral  changes  at  the 
muscle  level  or  an  inadequate  output  from  the  central 
nervous  system  to  stimulate  motoneurons  [4].  Intricacies  of 
muscle  fatigue  are  associated  with  several  aspects  where  the 
relative  importance  of  each  depends  on  the  fiber  type  and 
composition  of  the  contracting  muscle(s),  as  well  as  the 
intensity,  type,  and  duration  of  contraction  activity.  Muscle 
cells  are  the  focal  points  of  fatigue  and  rarely  involve  the 
central  nervous  system  or  the  neuromuscular  junction  [5]. 
The  amount  of  force  generated,  duration  of  each  contraction, 
and  the  rest  period  between  two  contractions  has  a  direct 
influence  on  the  muscle  fatigue  rate  [6].  Muscle  fiber-type 
distribution  [7,  8],  nerve  conduction  velocity  of  fatiguing 
muscles  [9,  10],  or  even  central  factors  within  the  central 
nervous  system  (CNS)  will  affect  motivation  to  perform 
activities  [11].  The  EMG  analysis  is  a  well-accepted  method 
for  muscle  fatigue  assessment  [8-10,  12-17].  Even  though 
the  sEMG  has  some  limitations  associated  with  the  skin 
impedance,  electrode  placement,  and  cross-talk,  it  is  used  for 
the  estimation  of  muscle  fatigue  of  different  muscles  [16,  18, 
19]. 

L.  Lindstrom  et  al.  developed  a  method  that  measures  the 
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localized  muscle  fatigue  based  on  the  power  spectrum 
analysis  using  myoelectric  signals,  [20].  This  approach 
permits  real-time  investigations  and  can  yield  statistically 
based  criteria  for  the  occurrence  of  fatigue.  Rate  of  fatigue 
development  and  changes  in  muscle  action  potential 
conduction  velocity  were  used  to  interpret  the  findings  [20- 
22].  Additional  recruitment  of  motor  units,  synchronization 
of  active  motor  units  along  the  muscle  fibers,  and  a  decrease 
in  conduction  velocity  is  reflected  in  the  EMG  signal  as  an 
increase  of  amplitude  in  time  domain  and  a  decrease  of 
medium  frequency  in  frequency  domain  [23,  24]. 

The  joint  analysis  method  using  sEMG  amplitude  and 
spectrum  (JASA)  allows  distinguishing  between  the 
difference  of  fatigue-induced  and  force  related  EMG 
changes.  Simultaneous  changes  in  the  EMG  amplitude  and 
spectrum  is  considered  in  the  JASA  approach  [25]. 
According  to  traditional  measurements,  the  EMG  amplitude 
increases  and  median  frequency  (MF)  decreases  as  a  result 
of  muscle  fatigue  [26-28].  Fatigue  can  occur  because  of 
continuous  high  frequency  and  tetanic  stimulations.  Decline 
in  the  force  magnitude  can  be  attributed  to  reduced  Ca++ 
release  from  the  sarcoplasmic  reticulum  (SR),  reduced 
myofibrilllar  Ca++  sensitivity,  or  because  of  reduced 
maximum  Ca++-  activated  tension.  The  main  reason  of  the 
tension  decline  with  continuous  tetanic  stimulation  is 
decreased  Ca++  release,  which  is  due  to  impaired  action 
potential  propagation  in  the  T  tubules.  Decrease  in  pH  and 
increase  in  inorganic  phosphate  (Pi)  concentration  causes 
reduced  Ca++  sensitivity  and  decline  in  maximum  tension. 
This  is  the  main  contributing  factor  in  decline  of  force  with 
continual  tetanic  stimulation  [29]. 

Increase  in  the  inorganic  phosphate  in  the  myoplasm 
([Pi]myo)  results  in  reduced  SR  Ca++  release  in  both 
skinned  and  intact  fibers.  Muscle  performance  declines  with 
rigorous  activities  which  results  in  fatigue.  Metabolic 
changes  on  either  the  contractile  machinery  or  the  activation 
process  are  also  responsible  for  the  fatigue  of  muscles. 
Myofibrillar  proteins  and  activation  process  both  are 
affected  during  fatigue  with  substantial  increase  in  the 
concentration  of  inorganic  phosphate  (Pi)  in  myoplasm. 
Further,  it  has  been  shown  that  failure  of  the  sarcoplasmic 
reticulum  (SR)  to  release  Ca++  also  contributes  to  fatigue 
[30].  During  intense  exercise  of  skeletal  muscles  (less  than 
20  seconds),  cells  consume  100  of  times  more  energy  than 
during  the  rest  period.  The  aerobic  capacity  of  muscle  cells 
falls  short  on  energy  demand  and  anaerobic  metabolism 
must  supply  the  majority  of  the  adenosine  triphosphate 
(ATP)  required.  Skeletal  muscle  fatigue  results  because  of 
the  high-intensity  exercise.  Hence,  the  anaerobic  metabolism 
pathway  results  in  a  decline  of  contractile  functionality  [31]. 

The  present  work  investigates  the  change  in  sEMG  in 
frequency  domain  during  skeletal  muscle  fatigue.  The  sEMG 
signals  are  acquired  for  multiple  subjects  for  dynamic  and 
static  force  experiments  to  induce  skeletal  muscle  fatigue. 
The  sEMG  signals  are  filtered  with  a  nonlinear  Teager- 
Kaiser  Energy  (TKE)  operator-based  nonlinear  spatial  filter 


[32].  Two  sets  of  dynamic  force  data  are  segmented  into 
three  and  five  parts  and  two  sets  of  static  force  data  are 
segmented  into  five  parts  each.  There  is  a  75%  overlap 
between  the  two  adjacent  segments.  A  number  of  classical, 
model-based  and  eigenvector  based  spectral  estimation 
techniques  are  used  to  study  the  change  in  the  sEMG  signals 
as  a  result  of  muscle  fatigue.  In  classical  methods  Fast 
Fourier  transform  (FFT)  and  Welch’s  averaged  modified 
periodogram  methods  are  used.  In  case  of  model-based 
methods  Yule-Walker  (Y-W),  Burg,  Covariance  (Cov.)  and 
Modified  Covariance  (Mcov.)  Autoregressive  (AR)  methods 
are  applied.  For  eigenvector  methods  Multiple  Signal 
Classification  (MUSIC)  and  Eigenvector  (EIG)  spectral 
estimation  methods  were  selected  for  processing  sEMG 
signals.  Using  these  spectrum  analysis  techniques,  Power 
Spectral  Density  (PSD)  estimates  and  detailed 
documentations  of  the  sEMG  signals  were  obtained.  These 
methods  were  compared  in  terms  of  their  frequency 
resolution  and  the  effects  in  determination  of  skeletal  muscle 
fatigue. 

II.  Experimental  Set-Up  And  Pre-Processing 

An  experiment  set-up  was  developed  using  DELSYS® 
Bagnoli-16  EMG  system  with  nine  DE-3.1  sEMG  sensors  to 
capture  the  sEMG  signals  from  skeletal  muscles  as  given  in 
[32].  This  arrangement  involves  nine  sensors  covering  four 
directional  spatial  distributions  of  the  sEMG  signal.  The 
appropriate  motor  point  of  the  flexor  digitorum  superficialis 


Fig.  1.  Experimental  set-up  for  dynamic  force  levels. 


muscle  (FDS),  which  controls  the  flexion  of  the  ring  finger, 
was  identified  using  a  wet  probe  muscle  stimulator  at  the 
FDS  (RICH-MAR,  HV  1000).  The  middle  three  sEMG 
sensors  were  attached  directly  on  the  skin  surface  above  the 
motor  point  of  the  ring  finger.  Prior  to  placing  the  sEMG 
sensors,  the  skin  surface  of  the  subject  was  prepared 
according  to  International  Society  of  Electrophysiology  and 
Kinesiology  (ISEK)  protocols.  Two  different  sets  of  fatigue 
inducing  experiments  were  conducted  using  this  set-up  of 
sEMG  sensors.  One  experiment  using  dynamic  force 
variations  and  another  with  a  50  pounds  of  static  force.  For 
the  dynamic  force  variation  we  used  an  Interlink™ 
Electronics  FSR  0.5”  circular  force  sensor  on  a  stress  ball 
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and  for  the  static  force  experiment  we  used  a  cable 
tensionmeter  (T5166)  by  ‘Pacific  Scientific  Company.’ 


Fig.  2.  Experimental  set-up  for  50  pounds  static  force  levels. 


For  the  dynamic  force  experiment  we  restricted  the  thumb 
movement  using  a  thumb  splint.  For  the  static  force 
experiment  we  held  the  force  of  the  dynamometer  at  50 
pounds  and  tried  to  maintain  this  force  level  to  induce 
fatigue  in  skeletal  muscles.  Force  data  for  dynamic  force 
experiments  was  captured  using  NI  ELVIS™  with 
Interlink™  Electronics  FSR  0.5”  circular  force  sensor. 
Experimental  set-up  is  shown  in  Fig.  1  and  2,  where  9 
sensors  are  shown  on  a  healthy  subject  forearm,  holding  a 
stress  ball  and  a  tension  dynamometer,  respectively.  The 
sEMG  and  finger  force  data  was  collected  at  a  sampling  rate 
of  2000  Hz  using  Lab  VIEW™  in  conjunction  with 
DELSYS®  Bagnoli-16  EMG  and  NI  ELVIS™.  With  these 
experimental  set-ups,  we  conducted  several  experiments  of 
30  seconds,  45  seconds  and  60  seconds  durations. 

III.  Spectral  Estimation  Methods 

Signals  can  be  analyzed  in  the  time  and  frequency 
domains  and  in  some  instances  the  frequency  content  of  the 
signal  is  more  useful  than  the  time  domain  characteristics 
[33].  Various  bio  signals  such  as  the  heart  rate,  EMG,  EEG, 
ECG,  eye  movements,  and  other  motor  responses,  acoustic 
heart  sounds,  and  stomach  and  intestinal  sounds,  show  much 
richer  information  in  the  frequency  domain  [33].  Spectral 
analysis  is  a  mathematical  prism  which  finds  the  frequency 
content  of  a  waveform  by  decomposing  the  signal  into  its 
constituent  frequencies  [33,  34].  There  is  a  wide  range  of 
methods  for  spectral  analysis,  each  having  its  own  benefits 
and  drawbacks.  In  this  research  we  are  using  classical 
methods  based  on  the  Fourier  transform,  modern  methods 
based  on  the  estimation  of  model  parameters,  and 
eigenvector  based  methods  [33]  in  order  to  characterize  the 
muscle  fatigue  occurring  in  skeletal  human  muscles,  in 
particular  muscles  of  the  forearm.  To  use  the  spectral 
analysis  wisely,  we  need  to  have  an  understanding  of  the 
spectral  features  of  interests  and  the  best  methods  to 
accurately  determine  those  features  [33]. 

A.  Discrete  Fourier  Transform  (DFT) 

DFT  which  is  the  computational  basis  of  the  spectral 
analysis  transforms  the  time  or  space  domain  data  into 


frequency  domain  data  [35].  The  DFT  of  a  vector  x  of  length 
N  is  given  as 

Hk)  =  E7=i*O0<4'_1)(fc_1),  (1) 

where  coN  =  e^~2m^N  is  the  Nth  root  of  unity  [35]. 

B.  Welch ’s  averaged  modified periodogram  method: 

As  the  name  suggests,  the  ‘Welch’s  averaged  modified 
periodogram  method’  depends  on  the  periodogram  of  the 
signal  {x(n)^}  which  is  given  by  Equation  (2),  [35]. 

SpeAD  =  ^  IXn=i  x(n)exp(—j2nfn)\2.  (2) 

In  Welch  method,  the  signal  is  segmented  into  eight  parts  of 
equal  length  with  an  overlapping  ratio  of  50%  and  each  part 
is  segmented  using  a  Hamming  window  as  given  by 
Equation  (3),  [35]. 

win)  =  0.54  -  0.46  cos  (2?r  ,  0  <  n  <  N  .  (3) 

C.  Yule-Walker  (Y-W)  autoregressive  (AR)  method: 

The  Yule-Walker  autoregressive  method,  also  called  the 
autocorrelation  method,  estimates  the  power  spectral  density 
(PSD)  of  the  input.  This  method  fits  an  autoregressive  (AR) 
model  to  the  windowed  input  data  by  minimizing  the 
forward  prediction  error  in  the  least-squares  sense.  This 
formulation  leads  to  the  Yule- Walker  equations,  which  are 
solved  by  Levinson-Durbin  recursion  [35]. 

D.  Burg  autoregressive  (AR)  method: 

The  Burg  autoregressive  (AR)  method  is  a  parametric 
spectral  estimation  method  of  the  signal,  x.  The  power 
spectral  density  is  calculated  in  units  of  power  per  radians 
per  sample.  This  method  is  based  on  the  minimization  of  the 
forward  and  backward  prediction  error  and  on  estimation  of 
the  reflection  coefficients  [35]. 

E.  Covariance  (Cov.)  autoregressive  (AR)  method: 

The  covariance  autoregressive  (AR)  method  uses  the 
covariance  algorithm  to  estimate  the  parametric  spectral 
density  of  the  signal,  x.  Based  on  causal  observation  of  the 
input  signal,  the  covariance  method  minimizes  the  forward 
prediction  error  and  fits  an  AR  linear  prediction  filter  model 
to  the  signal  [35]. 

F.  Modified  covariance  (Mcov.)  autoregressive  (AR) 
method: 

Modified  covariance  autoregressive  (AR)  method  estimates 
the  PSD  of  the  signal  using  the  modified  covariance  method. 
Based  on  the  causal  information  of  the  input  signal,  the 
modified  covariance  method  fits  an  autoregressive  (AR) 
linear  prediction  filter  model  to  the  signal  by  simultaneously 
minimizing  the  forward  and  backward  prediction  errors.  The 
spectral  estimate  returned  by  this  method  is  the  magnitude 
squared  frequency  response  of  the  AR  model  [35]. 

G.  Multiple  Signal  Classification  (MUSIC)  spectral 
estimation  method: 

The  MUSIC  algorithm  estimates  the  pseudospectrum  (in 
rad/sample)  at  the  corresponding  vector  of  frequencies  for 
the  input  signal  x  [35].  This  algorithm  uses  the  estimates  of 
the  eigenvectors  of  a  correlation  matrix  associated  with  the 
input  signal  using  Schmidt’s  eigenspace  analysis  method 
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[36].  The  MUSIC  pseudospectrum  estimate  is  given  by 
Equation  (4), 


:(/)  = 


_ _ _ = _ _ _  (4) 

eHm(Z%=p+1vkv»)em  ^=p+1|v«e(/)|2’  v  ; 

where  N  is  the  dimension  of  the  eigenvectors  and  vk  is  the 
k- th  eigenvector  of  the  correlation  matrix  [35].  The  signal 
subspace  has  a  dimension  p  and  the  eigenvectors  vk  used  in 
the  sum  corresponds  to  the  smallest  eigenvalues  and  also 
spans  the  noise  subspace  [35].  The  vector  e(/)  consists  of 
the  complex  exponentials,  so  the  inner  product  vke(f ) 
amounts  to  a  Fourier  transform.  To  estimates  the 
pseudospectrum,  the  squared  magnitudes  are  summed  for 
FFT  computed  for  each  vk  [35]. 


H.  Eigenvector  (EIG)  spectral  estimation  method: 

The  eigenvector  spectral  estimation  method  estimates  the 
pseudospectrum  (in  rad/sample)  at  the  corresponding  vector 
of  frequencies  using  estimates  of  the  eigenvectors  of  a 
correlation  matrix  associated  with  the  input  signal  x  [35]. 
This  method  estimates  the  pseudospectrum  from  a  signal  or 
a  correlation  matrix  using  a  weighted  version  of  the  MUSIC 
algorithm  derived  from  Schmidt’s  eigenspace  analysis 
method  [36,  37].  To  find  the  frequency  content  of  the  signal 
the  algorithm  performs  eigenspace  analysis  of  the  signal’s 
correlation  matrix.  Singular  value  decomposition  is  used  to 
compute  the  eigenvalues  and  eigenvectors  of  the  signal’s 
correlation  matrix  [35].  This  method  computes  the 
pseudospectrum  estimate  as  given  by  Equation  (5). 


pev(.n  = 


(5) 


G"=p+iI^(/)|  )Mk 
where  the  eigenvectors  have  a  dimension  of  N  and  vk  is  the 
k- th  eigenvector  of  the  correlation  matrix  [35].  The  signal 
subspace  has  a  dimension  p  and  the  eigenvectors  vk  used  in 
the  sum  corresponds  to  the  smallest  eigenvalues  and  also 
spans  the  noise  subspace  [35].  The  vector  e(/)  consists  of 
the  complex  exponentials,  so  the  inner  product  vke(f) 
amounts  to  a  Fourier  transform  and  to  estimate  the 
pseudospectrum,  the  squared  magnitudes  are  summed  for 
FFT  computed  for  each  vk  [35]. 


I.  Selection  of  Model  Orders  for  Model-Based  and 
Eigenvector  Based  Methods 

Model-based  and  eigenvector-based  methods  need  to  have 
a  specific  model  order  which  is  an  important  aspect  of  the 
use  in  these  methods.  Using  the  sEMG  and  force  data  as 
input  and  outputs  for  three  and  five  sets  of  the  segments  for 
different  data  sets,  we  created  model  structure  matrices  using 
‘struc’  function  in  MATFAB®,  using  ‘arxstruc’  we 
compared  a  model  order  of  1  to  50th  with  varying  delay  of  1 
to  50  using  cross-validation  on  the  second  half  of  the  data 
set.  With  this  approach,  it  was  possible  to  select  the  order 
that  gives  the  best  fit  for  the  validation  data  set. 


averaged  modified  periodogram,  Yule- Walker,  Burg, 
Covariance,  Modified  Covariance  autoregressive  (AR), 
Multiple  Signal  Classification  (MUSIC)  and  Eigenvector 
spectral  estimation  methods.  The  objective  of  this  study  was 
to  determine  preferred  methods  of  signal  processing  that 
elevates  the  sensitivity  of  muscle  fatigue  as  represented  in 
the  PSD  of  the  sEMG  signal.  An  increased  sensitivity  allows 
for  better  modeling  of  the  fatigue  phenomena  and  hence 
more  accurate  sEMG  models.  Ultimately  this  may  lead  to 
better  prosthetic  control. 

Data  of  two  experiments  for  dynamic  force  variations  was 
segmented  in  three  and  five  parts  respectively.  Each  segment 
is  with  an  overlap  of  75%  with  its  adjacent  segment.  Using 
different  methods,  we  computed  the  PSDs  for  each  segment. 
For  the  dynamic  force  experiments,  the  maximum  value  of 
PSDs  of  sEMG  signal  increases  with  muscle  fatigue  as  time 
or  segment  number  is  increased.  The  classical  methods  (FFT 
and  Welch)  and  eigenvector  based  methods  (MUSIC  and 
Eigenvector  (EIG.))  are  representing  this  change  well  in  case 
of  maximum  PSD  values  and  show  a  clear  difference. 

TABLE  I 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  AND 
EIGENVECTOR  BASED  METHODS  -  DYNAMIC  VARYING  FORCE  - 
EXPERIMENT  2 


Segment  No. 

Classical-Methods 

Eigenvector-Methods 

FFT 

Welch 

MUSIC 

EIG 

1st 

4.1e+6 

0.0082 

371.59 

1.0819 

2nd 

5.09e+6 

0.0100 

424.43 

1.3892 

3rd 

5.74e+6 

0.0111 

480.71 

1.4132 

4th 

6.79e+6 

0.0133 

508.64 

1.4936 

5th 

2.13e+7 

0.0379 

695.08 

10.2118 

Table  I  lists  the  peak  values  of  the  PSDs  of  five  segments 
using  classical  and  eigenvector  based  methods  for  a 
dynamically  varying  force  experiment.  Fig.  1  shows  the 
overlapping  plot  of  PSDs  for  five  segments  using  the 
MUSIC  algorithm  based  spectral  estimation  method.  The 
increase  in  the  maximum  PSD  value  is  evident  from  the  1st 
to  the  5th  segment  of  the  data. 


MUSIC  -  Dynamic  Force  -  Experiment  2 


Fig.  3.  PSD  vs.  Frequency  Plot  for  MUSIC  Algorithm  - 
Dynamic  Force  Experiment. 


IV.  Results  and  Discussion 

sEMG  signal  changes  as  a  consequence  of  muscle  fatigue 
[23-28],  the  amplitude  of  the  PSD  of  the  signal  increases  and 
the  median  frequency  shifts  towards  the  lower  frequency 
range  [26-28].  In  this  study,  PSDs  of  the  different  segments 
of  each  sEMG  data  set  were  obtained  using  FFT,  Welch’s 


Fig.  3  shows  the  resulting  PSD  using  the  Burg  method. 
Comparing  Fig.  3  and  4,  the  progression  of  fatigue  influence 
shift  in  PSDs  is  evident  in  both  plots.  However,  the  MUSIC 
algorithm  not  only  shows  larger  amplitudes,  but  also  a 
greater  relative  sensitivity  to  fatigue.  The  rather  equal 
spacing  between  the  lines  of  the  PSD  for  the  MUSIC 
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algorithm  compared  to  the  Burg  method  indicates  a  rather 
more  linear  relationship  of  the  fatigue  progression. 


Burg  -  Dynamic  Force  -  Exp88 
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Fig.  4.  PSD  vs.  Frequency  Plot  for  Burg  Method  -  Dynamic 
Force  Experiment. 


segments  of  a  static  force  experiment.  While  providing  a 
large  maximum  value,  the  FFT  method  is  limited  by  its  own 
spectral  resolution  (1  /N)  and,  due  to  windowing  of  the  finite 
data  set,  results  into  spectral  leaking. 


0.5  1  1.5  2  2.5 

Frequency  (radian/ sample)  x  104 


Fig.  5.  PSD  vs.  Frequency  Plot  for  Eigenvector  Method  - 
Static  Force  of  50  Pounds. 


TABLE  II 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  - 
STATIC  FORCE  -  50  POUNDS  -  EXPERIMENT  1 


Segment  No. 

Classical-Methods 

Eigenvector-Methods 

FFT 

Welch 

MUSIC 

EIG 

1st 

1.03e+5 

6.81e-5 

1413 

9.93e-4 

2nd 

1.80e+5 

1.12e-4 

1793 

16e-4 

3rd 

3.21e+5 

1.80e-4 

2296 

28e-4 

4th 

4.25e+5 

2.62e-4 

3102 

37e-4 

5th 

6.23e+5 

3.58e-4 

7723 

104e-4 

TABLE  III 

MAXIMUM  VALUE  OF  PSD  FOR  CLASSICAL  METHODS  - 
STATIC  FORCE  -  50  POUNDS  -  EXPERIMENT  1 


Segment  No. 

1st 

2nd  2r(* 

4th 

5th 

Model-Based 

Methods 

7.7e-7 

1.3e-6  2.3e-6 

3.1e-6 

7.81-6 

FFT  -  Static  Force  -  50  Pounds  -  1st  and  5th  Segment  -  Exp33 


Frequency  (Hz) 


Fig.  6.  PSD  vs.  Frequency  Plot  for  FFT  Method  -  Static 
Force  of  50  Pounds. 


The  sEMG  data  of  two  experiments  for  static  force  (50 
pounds)  were  processed  and  the  maximum  PSDs  of  five 
segments  using  classical,  model-based,  and  eigenvector 
based  methods  were  computed.  Data  from  both  the 
experiments  show  similar  results  as  the  dynamic  case.  Table 
II  lists  the  peak  values  of  the  PSDs  of  five  segments  using 
classical  and  eigenvector  based  methods  for  static  force  (50 
pounds)  for  one  experiment.  Table  III  lists  the  maximum 
values  of  the  PSD  for  model-based  methods:  Y-W,  Burg, 
Covariance,  and  modified  covariance.  All  of  these  methods 
resulted  in  the  same  maximum  values  for  each  segment. 
Comparing  Table  II  and  III,  we  recognize  the  large 
difference  in  maximum  value  between  the  model-based 
methods  and  the  corresponding  values  from  the  FFT  and 
MUSIC  method.  Fig.  5  shows  the  overlapping  plot  of  PSDs 
for  five  parts  using  eigenvector  algorithm  based  spectral 
estimation  method.  The  increase  in  the  maximum  PSD  value 
is  evident  from  the  1st  to  the  5th  segment  of  the  static  force 
sEMG  data. 

The  eigenvector  method  produces  a  similar  characteristic 
as  the  MUSIC  algorithm  and  distinguishes  itself  by  also 
providing  a  more  linear  characteristic  of  the  fatigue 
progression  and  a  greater  relative  sensitivity.  Fig.  6  depicts 
the  PSD  generated  by  using  FFT  method  for  the  five 


All  the  model-based  methods  for  both  dynamic  and  static 
force  levels  produce  the  same  peak  values  and  the  same  PSD 
for  the  corresponding  experiment.  Since  Burg  and  Y-W 
methods  guarantee  stability  while  the  covariance  and 
modified  covariance  methods  have  conditions  for  stability  to 
be  satisfied  (i.e.  min.  order  must  be  of  certain  length  of  the 
input  frame  size),  Burg  and  Y-W  should  be  the  preferred 
methods  for  sEMG  analysis.  However,  the  Burg  method  is  to 
be  preferred  if  short  data  sets  are  used. 

Comparing  the  eigenvector  based  methods  (MUSIC  and 
Eigenvector),  both  of  these  methods  are  frequency  estimator 
techniques  based  on  eigenanalysis  of  the  autocorrelation 
matrix  where  the  resulting  estimate  has  sharp  peaks  at  the 
frequencies  of  interest.  The  eigenvector  method  uses  inverse 
eigenvector  weighting  whereas  the  MUSIC  method  uses 
unity  weighting,  implying  that  the  eigenvector  method  gives 
fewer  spurious  peaks  than,  the  MUSIC  algorithm  [38].  As 
seen  from  the  dynamic  experiment  results,  the  MUSIC 
method  provides  for  a  better  spacing  between  the  segment 
PSDs  compared  to  the  Burg  and  eigenvector  method.  From 
static  experiments,  we  conclude  that  all  three  (MUSIC, 
Burg,  and  Eigenvector)  methods  perform  similarly  if  a  linear 
relationship  of  the  fatigue  progression  is  desired. 
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V.  Conclusion  and  Future  Work 

In  this  paper,  we  have  shown  the  skeletal  muscle  fatigue 
related  change  in  the  sEMG  signal  in  the  frequency  domain. 
Classical  (fast  Fourier  transform  and  Welch’s  averaged 
modified  periodogram),  model-based  (Yule-Walker,  Burg, 
Covariance,  and  Modified  Covariance  autoregressive  (AR) 
method)  and  eigenvector  based  methods  (Multiple  Signal 
Classification  (MUSIC)  and  eigenvector  spectral  estimation 
method)  are  used  to  compute  the  PSDs.  Classical  and 
eigenvector  based  methods  are  more  sensitive  than  the 
model-based  methods  for  analyzing  the  fatigue  related 
changes  in  sEMG  signal.  However,  the  MUSIC  algorithm 
provides  good  maximum  value  in  the  PSD  as  well  as  a  clear 
distinction  between  the  segmented  sEMG  data.  The  latter 
point  is  indicative  of  a  relative  linear  fatigue  progression  in 
time  for  the  MUSIC  algorithm.  In  future  work  these  results 
can  be  used  to  design  and  improve  the  skeletal  muscle 
‘Force-sEMG-Fatigue’  based  models  [39]  for  prosthetic 
design  and  other  rehabilitation  research. 
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Abstract  -  Skeletal  muscle  force  can  be  estimated  using 
surface  electromyographic  (sEMG)  signals.  Usually,  the 
surface  location  for  the  sensors  is  near  the  respective  muscle 
motor  unit  points.  Skeletal  muscles  generate  a  spatial  EMG 
signal,  which  causes  cross  talk  between  different  sEMG  signal 
sensors.  In  this  study,  an  array  of  three  sEMG  sensors  is  used 
to  capture  the  information  of  muscle  dynamics  in  terms  of 
sEMG  signals.  The  recorded  sEMG  signals  are  filtered 
utilizing  optimized  nonlinear  Half-Gaussian  Bayesian  filters 
parameters,  and  the  muscle  force  signal  using  a  Chebyshev 
type-II  filter.  The  filter  optimization  is  accomplished  using 
Genetic  Algorithms.  Three  discrete  time  state-space  muscle 
fatigue  models  are  obtained  using  system  identification  and 
modal  transformation  for  three  sets  of  sensors  for  single 
motor  unit.  The  outputs  of  these  three  muscle  fatigue  models 
are  fused  with  a  probabilistic  Kullback  Information  Criterion 
(KIC)  for  model  selection.  The  final  fused  output  is  estimated 
with  an  adaptive  probability  of  KIC,  which  provides 
improved  force  estimates. 


I.  Introduction 

National  Limb  Loss  Information  Center  [1]  reported  in 
2002  that  1.2  million  people  live  with  amputations.  Since 
then,  this  number  is  increasing  due  to  ongoing  wars  in  Iraq 
and  Afghanistan.  However,  prostheses  can  significantly 
improve  the  lives  for  these  people. 

Recent  research  efforts  have  been  active  towards 
creating  surface  electromyographic  (sEMG)  based 
prosthetics  for  hand  amputees.  The  recorded  sEMG  signal 
is  used  as  an  input  to  activate  the  prosthesis.  The  central 
nervous  system  activates  and  controls  the  EMG  signals 
which  depend  on  the  flow  of  specific  ions  including 
sodium  (Aa+),  potassium  ( K+ )  and  calcium  ( Ca++ ) 
resulting  in  the  action  potentials  in  nerves  and  their 
respective  skeletal  muscle  fibres.  A  potential  difference 
develops  across  neuronal  membranes  because  of  this  ion 
exchange  and  this  can  be  measured  as  an  electrical  voltage 

Parmod  Kumar  is  with  Measurement  and  Control  Engineering 
Research  Center  (MCERC),  College  of  Engineering,  Idaho  State 
University,  Pocatello,  Idaho  83209,  USA  (email:  kumaparm@isu.edu). 

Anish  Sebastian  is  with  MCERC,  College  of  Engineering,  Idaho  State 
University,  Pocatello,  Idaho  83209,  USA  (email:  sebaanis@isu.edu). 

Chandrasekhar  Potluri  is  with  MCERC,  College  of  Engineering, 
Idaho  State  University,  Pocatello,  Idaho  83209,  USA  (e-mail: 
potlchan@isu.edu). 

Alex  Urfer  is  with  Department  of  Physical  and  Occupational 
Therapy,  Idaho  State  University,  Pocatello,  Idaho  83209,  USA  (email: 
urfealex@isu.edu). 

D.  Subbaram  Naidu  is  with  Department  of  Electrical  Engineering, 
MCERC,  Idaho  State  University,  Pocatello,  Idaho  83201  USA  (email: 
naiduds@isu.edu). 

Marco  P.  Schoen  is  with  Department  of  Mechanical  Engineering, 
MCERC,  Idaho  State  University,  Pocatello,  Idaho  83209,  USA  (email: 
schomarc@isu.edu). 


change  [2].  sEMG  signals  are  collected  from  the  skin  as  an 
electric  voltage  ranging  between  -5  and  +5  mV.  The 
control  of  the  myoelectric-based  prosthesis  is  difficult 
because  of  the  dynamic  nature  of  the  EMG  signal  [1]  due 
to  varying  motor  unit  recruitment,  crosstalk,  and 
biochemical  interaction  within  the  muscular  fibres.  In 
addition,  sEMG  changes  continuously  due  to  the  onset  and 
progression  of  muscle  fatigue  [1]. 

Muscle  fatigue  is  complex  in  nature  and  results  in 
failure  to  maintain  the  required  force  level  [3].  Reasons  for 
fatigue  can  be  the  peripheral  changes  at  the  muscle  level  or 
an  inadequate  output  from  the  central  nervous  system  to 
stimulate  motoneurons  [3].  The  amount  of  force  generated, 
duration  of  each  contraction,  and  the  rest  period  between 
two  contractions  has  a  direct  influence  on  the  muscle 
fatigue  rate  [4].  Muscle  fibre-type  distribution  [5],  nerve 
conduction  velocity  of  fatiguing  muscles,  or  even  factors 
within  the  central  nervous  system  (CNS)  affect  EMG 
signals  [6].  EMG  analysis  is  a  well-accepted  method  for 
muscle  fatigue  assessment  [6]. 

In  1977,  L.  Lindstrom,  R.  Kadefors  and  I.  Petersen 
developed  a  method  that  measures  the  localized  muscle 
fatigue  based  on  the  power  spectrum  analysis  using 
myoelectric  signals  [7].  This  approach  permits  real-time 
investigations  and  can  yield  statistically  based  criteria  for 
the  occurrence  of  fatigue.  Rate  of  fatigue  development  and 
changes  in  muscle  action  potential  conduction  velocity 
were  used  to  interpret  the  findings  [7].  Additional 
recruitment  of  motor  units,  synchronization  of  active  motor 
units  along  the  muscle  fibres,  and  a  decrease  in  conduction 
velocity  are  reflected  in  the  EMG  signal  as  an  increase  of 
amplitude  in  time  domain  and  a  decrease  of  medium 
frequency  in  frequency  domain  [8]. 

The  joint  analysis  method  using  EMG  amplitude  and 
spectrum  (JASA)  allows  distinguishing  between  the 
difference  of  fatigue-induced  and  force  related  EMG 
changes.  Simultaneous  changes  in  the  EMG  amplitude  and 
spectrum  are  considered  in  the  JASA  approach  [9].  The 
JASA  principle  states  that  muscle  force  decreases  because 
of  the  decrease  in  EMG  amplitude  [9].  Fatigue  can  occur 
because  of  continuous  high  frequency  stimulation  or 
because  of  titanic  stimulation.  The  Flow  Chart  of  the  work 
in  this  paper  is  shown  in  Figure  1. 

This  paper  addresses  the  issue  of  skeletal  muscle 
fatigue  that  is  a  dynamic  phenomenon.  The  modeling  is 
based  on  system  identification,  where  mathematical 
relations  are  inferred  from  experimental  data.  The  data 
consists  of  sEMG  signals  and  hand/fmger  force  generated 
by  healthy  subjects.  The  recorded  three-sEMG  signals  are 
filtered  using  nonlinear  Half-Gaussian  Bayesian  filters 
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with  optimized  filter  parameters,  whereas  the  skeletal 
muscle  force  signal  is  pre-processed  using  a  Chebyshev 
type-II  filter  [1,  10]. 


Force 

Signal 


sEMG 
Sensor  1 


Chebyshev  type-II 

filter 

H.G.  Bayesian  Filter  with 
Optimized  a  1  and  (31 
using  Genetic  Algorithm 


sEMG 

Sensor-M 


sEMG 

Sensor2 


H.G.  Bayesian  Filter  with 
Optimized  aM  and  (3M 
using  Genetic  Algorithm 


H.G.  Bayesian  Filter  with 
Optimized  a2  and  (32 
using  Genetic  Algorithm 


Figure  1 :  The  Flow  Chart  of  the  Work  in  this  Paper. 

A  simple  Genetic  Algorithm  code  is  used  to  optimize 
the  Bayesian  filter  parameters.  Using  an  input/output 
approach,  the  EMG  signal  is  the  input  to  the  skeletal 
muscle,  and  the  produced  hand/fmger  force  constitutes  the 
output.  Three  discrete  time  state-space  models  are  obtained 
using  system  identification  and  modal  transformation  for 
three  sets  of  sensors.  One  sensor  is  placed  at  the  motor  unit 
of  the  ring  finger  of  the  dominant  hand  and  two  sensors  are 
located  in  its  vicinity  as  shown  in  Figure  2.  The  extracted 
models  from  the  data  sets  are  fused  with  a  probabilistic 
Kullback  Information  Criterion  (KIC)  for  model  selection. 
The  final  fused  output  is  estimated  with  an  adaptive 
probability  of  KIC  that  gives  an  improved  skeletal  muscle 
force  estimates. 


II.  Experimental  set-up  and  pre-processing 
Both  sEMG  and  muscle  force  signals  were  acquired 
simultaneously  using  Lab  VIEW™  8.2  at  a  sampling  rate  of 
2000  Hz.  The  sEMG  data  capturing  was  aided  by  a 
DELSYS®  Bagnoli-16  EMG  system  with  DE-2.1 
differential  EMG  sensors.  The  corresponding  force  data 
was  captured  using  NI  ELVIS  with  Interlink  Electronics 
FSR  0.5”  circular  force  sensor.  The  experimental  set-up  is 
shown  in  Figure  2.  One  sEMG  sensor  was  placed  on  the 


motor  point  of  ring  finger  and  two  adjacent  to  the  motor 
point  of  a  healthy  subject.  Prior  to  placing  the  sEMG 
sensors,  the  skin  surface  of  the  subject  was  prepared 
according  to  International  Society  of  Electrophysiology 
and  Kinesiology  (ISEK)  protocols. 


Figure  2:  Experimental  Set-Up. 


The  EMG  signal  is  modeled  for  the  conditional 
probability  of  the  rectified  EMG  signal  as  a  filtered 
random  process  with  random  rate.  The  likelihood  function 
for  the  rate  evolves  in  time  according  to  a  Fokker-Planck 
partial  differential  Equation  [10].  Here,  a  and  /?  are  two 
free  parameters,  a  is  the  expected  rate  of  gradual  drift  in 
the  signal,  and  /?  is  the  expected  rate  of  sudden  shifts  in  the 
signal.  These  two  free  parameters  of  the  non-linear  “Half- 
Gaussian  filter  model”  are  optimized  for  the  acquired  EMG 
data  using  an  elitism  based  Genetic  Algorithm.  A 
Chebyshev  type  II  low  pass  filter  with  a  550  Hz  pass 
frequency  is  used  to  filter  the  force  signal. 


III.  System  identification  and  modal 

TRANSFORMATION 

An  alternative  to  physically  based  mathematical 
modeling  is  data  based  -‘system  identification  (SI)’-  that 
can  be  applied  to  different  systems  where  it  needs 
sufficient  experimental  data  [11].  The  parametric  system 
identification  process  yields  a  dynamical  model  such  as  the 
Auto  Regressive  with  exogenous  input  (ARX)  or  Output 
Error  (OE)  model,  this  model  gives  us  the  system  in 
mathematical  form.  In  this  work,  the  SI  approach  is  used 
with  the  myolelectric  signal  as  the  input  to  the  system  and 
as  the  output  is  the  intended  fmger/hand  force.  In  this 
work,  we  are  using  an  output  error  (OE)  model  structure 
for  the  modeling  of  EMG  and  hand/fmger  force  signals, 
which  has  been  documented  in  the  literature  to  perform 
well  for  EMG  signals  among  linear  parametric  models  [1]. 
Each  OE  model  takes  the  form  of  Equation  (1), 

y(0  =j^u(.t-nk)  +  e(t),  (1) 

where  t  is  time  index,  y(t)  is  output,  u(t  —  nk)  is  input, 
nk  is  the  delay,  e(t)  is  the  error,  and  B(q)  and  F(q)  are 
polynomials. 

A  common  OE  model  is  obtained  for  each  sensor  data 
set  using  one  set  of  model  parameters  (polynomial  order). 
The  OE  models  then  are  converted  into  a  state-space 
formulation.  By  doing  this,  the  identification  properties  of 
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the  OE  model  formulation  are  preserved  and  represented  in 
the  state-space  form.  A  modal  form  is  computed  by  using 
state-space  matrices  and  following  transformation: 

A  =  T-1AT,  H  =  T-1B,  T  =  CT,  (2) 

where  T  is  the  matrix  composed  of  the  eigenvectors  of  A, 
A ,  H  and  T  are  corresponding  transformations  for  matrix 
A,  B,  and  C.  Matrix  D  came  out  to  be  a  zero  matrix  for  the 
cases  treated  in  this  work.  Equation  (2)  refers  to  the 
continuous  representation  of  the  identified  models.  Using 
the  modal  form  representation  for  each  corresponding  data 
set,  the  influence  of  fatigue  can  be  observed  by  charting 
the  modal  coefficients  in  time  for  each  matrix.  For 
example,  the  coefficients  contained  in  the  transformed 
matrix  A  can  be  plotted  in  the  complex  s-plane.  This 
tracking  can  then  be  used  as  the  basis  for  constructing 
polynomials  that  connect  each  element  of  each  matrix  in 
time  with  the  corresponding  time  stamp.  Hence,  a  fatigue 
model  in  state-space  description  is  constructed,  where  the 
input  is  the  recorded  EMG  signal  and  the  output  the 
expected  force  generated  by  the  hand  motion  all  as  a 
function  of  time. 


4)  Compute  the  model  probability  p(Mt\Z)  =  — — — , 

£/= ie  J 

where  l  is  model  selection  criterion,  i.e.  KIC(pi). 

5)  Compute  the  fused  model  output  Yf  =  Yi1i=iV(Mi\Z)Yi. 

6)  Compute  the  overall  OE  model  from  Yf  and  force  data. 

Here  all  the  computation  from  step  2)  to  step  6)  is  adaptive 
i.e.  the  residual  square  norm,  A7C(pj),  model  probability 
p(Mj|Z),  and  fused  model  output  Yf  are  being  updated  with 
time  or  for  each  data  point. 

V.  Results,  conclusion  and  future  work 

Based  on  previous  research  results  and  after  trying 
different  model  orders  for  sEMG/Force  data,  an  optimal 
model  order  of  nb  =  2,  Uf  =  2,  nk  =  1  is  used  in  this 
work.  Equation  (4),  (5),  and  (6)  give  three  resulting 
discrete  time  state-space  sEMG-Force-Fatigue  models  for 
motor  point  sensor,  sensor- 1  and  sensor-2  respectively. 

*77i  (ft  +  1)  =  ^7n*7n  (ft)  +  BmEMGm(h); 

YmQi)  =  CmxmQi)  +  DmEMGm{h).  (4) 


IV.  Fusion  of  outputs  and  adaptive  probability 

OF  KIC 

Multiple  model  fusion  is  done  by  assigning  a  particular 
probability  to  each  individual  model  [12].  These  models 
are  obtained  using  SI  for  three  sEMG  sensors  for  single 
motor  unit.  SI  model  fit  value  is  used  to  compute  the 
probability  for  each  model.  The  fit  value  is  given  by 

[|T-f|1 

1  —  - — —  *  100.  The  model  selection  criterion  used  in 

\Y-Y\\ 

this  paper  is  Kullback  information  criterion  (KIC),  which 
is  an  asymmetric  measure.  The  sum  of  two  directed 
divergences,  which  is  the  measure  of  the  models 
dissimilarity,  is  known  as  Kullback’s  symmetric  or  J- 
divergence  [13],  as  given  by  Equation  (3). 

KIC(pd=^\ogRi+^^-nxp^)  +  g(n),  (3) 

where  g(n )  =  n  *  log(n/2). 

Following  fusion  technique  is  applied  for  sEMG  - 
force  identification  models,  which  is  based  on  [12]. 

1)  Identify  OE  models  Mlf  M2, ...  ,Mkl  using  sEMG  data 
( u )  as  input  and  force  data  (Y)  as  output,  for  kl  number  of 
sensors  collecting  data  simultaneously. 

2)  Compute  the  residual  square  norm 

st  =  ||y-«i»t§,||2  =  ||y-f||2, 

where  and 
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3)  Calculate  the  model  criteria  coefficient  using  Equation 

(3). 


x±(h  +  1)  =  A^Qi)  +  B^MG^h); 

Yt(h)  =  C^Qi)  +  D1EMG1(h).  (5) 

x2{h  +  1)  =  A2x2{K)  +  B2EMG2(h ); 

Y2  (h)  =  C2x2  (h)  +  D2EMG2  (h).  (6) 


In  these  case  matrices  Dm,D1)  and  D2  are  zero  matrices, 
whereas  matrices  Am,  Bm,  C{ 

I  a7ni  (0  0 

0  dm2 


77i,  i^TTi,  u-tti,  Ai,  Ri,  Ci,  ^2;  ^2 ?  and  G2  are 


given  as:  Am  = 
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,  A2,  B2,  and  C2 


r 

m>  u77i 


are 

function  of  time  and  the  elements  of  matrices  A1,B1,  and 
C1  are  constants  for  this  particular  data  set.  The  time 
variables  cimi  (t),  CLm 2 (t)»  ^mi  ^m2 

a2i(t).«22 (0.^21  OU22 (t),c21(t),  and  c22(t)  are  given 
by  quadratic  polynomials  obtained  by  curve  fitting  the 
tracked  modal  transformed  coefficients.  These  time 
variables  are  given  as: 


aml (t)  =  0.00026t2  -  0.033 1  -  11; 
am2  (f)  =  -0.0018t2  +  0.24t  —  33; 
bml  (t)  =  — 0.002t2  +  0.33t  +  42; 
bm2  (f )  =  — 0.0017t2  +  0.33 1  +  25; 
cmi(t)  =  (3.2  *  10-7)t2  -  (4.2  *  10"5)t  +  0.71; 
cm2(t )  =  (4.6  *  10-8)t2  -  (5.8  *  10-6)t  -  0.71; 
a2i(t)  =  -(1.2  *  10-5)t2  +  0.0013t  -  0.47; 
a22(t)  =  O.OOlf2  -  0.037t  -  35; 
b21(t)  =  (6.4  *  10-6)t2  -  O.OOlt  +  0.3; 
b22(t)  =  0.0023f2  -  0.12t  -  75; 
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c21(t)  =  —(1.8  *  10  7)t2  +  (6.4  *  10  6)t  +  0.71;  and 
c22(t)  =  -(2.1  *  10-9)t2  +  (2.4  *  10_7)t  -  0.71. 
Elements  of  matrices  A1,B1,  and  C1  are  constants  for  this 
particular  data  set,  they  are  given  as: 

an  =  —0.0159;  a12  =  —43.1883;  b1±  =  0.0090; 

b12  =  -72.1875;  c±1  =  0.7147;  and  c12  =  -0.7071. 

The  discrete  models  have  a  sampling  time  of  At  =  0.0005 
seconds.  The  duration  of  the  final  estimated  and  fused 
output  for  three  sensors  is  37.36  seconds.  Figure  3  depicts 
the  probability  and  10th  degree  curve  fitting  for  motor 
point,  ring2  and  ringl  sensor  signals. 
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Figure  3:  10th  Degree  Curve  Fitting  -  Probability  Motor, 
Ring2  and  Ringl  Sensor  Signals. 


Measured  Output  Vs.  Estimated  Fused  Output  with  Adaptive  Probability 


Figure  4:  Measured  Output  Vs.  Estimated  Fused  Output 
with  Adaptive  Probability. 


Validation  Plot  -  Measured  Output  Vs.  Estimated  Fused  Output  With  Adaptive  Probability 
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Time  (37.36  secs)  xio4 

Figure  5:  Validation  Plot  -  Measured  Vs.  Estimated  Fused 
Output  with  Adaptive  Probability. 


Figure  4  and  Figure  5  shows  the  measured  and 
estimated  fused  output,  depending  on  the  error  between  the 
two  later  follows  the  measured  signal  very  closely.  This 
approach  identifies  a  fused  fatigue  model  and  estimates  an 
adaptive  probability  based  output  that  follows  the 
measured  output  very  closely.  The  outcome  of  this  research 
can  be  used  to  estimate  the  skeletal  muscle  force  of  a 
human  hand  for  prosthetic  hand  design,  application  and 
improvement. 

In  future  work  improvement  of  the  data  collection 
techniques  and  sEMG  sensor  design,  may  lead  to  improved 
force  estimation. 
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Abstract:  -  Skeletal  muscle  force  can  be  estimated  using  surface  electromyographic  (sEMG)  signals.  Usually,  the 
sEMG  location  for  the  sensors  is  near  the  respective  muscle  motor  unit  points.  Skeletal  muscles  generate  a  temporal 
and  spatial  distributed  EMG  signal,  which  causes  cross  talk  between  different  sEMG  signal  sensors.  In  this  paper,  an 
array  of  three  sEMG  sensors  is  used  to  capture  the  information  of  muscle  dynamics  in  terms  of  sEMG  signals  and 
generated  muscle  force.  The  recorded  sEMG  signals  are  filtered  utilizing  optimized  nonlinear  Half-Gaussian  Bayesian 
filter,  and  a  Chebyshev  type-II  filter  prepares  the  muscle  force  signal.  The  filter  optimization  is  accomplished  using 
Genetic  Algorithm  (GA).  Multi  nonlinear  Auto  Regressive  exogenous  (ARX)  and  Wiener-Hammerstein  models  with 
different  nonlinearity  estimators/classes  are  obtained  using  system  identification  (SI)  for  three  sets  of  sensor  data.  The 
outputs  of  these  models  are  fused  with  a  probabilistic  Kullback  Information  Criterion  (KIC)  for  model  selection  and  an 
adaptive  probability  of  KIC.  First,  the  outputs  are  fused  for  the  same  sensor  and  for  different  models  and  then  the  final 
outputs  from  each  sensor.  The  final  fusion  based  output  of  three  sensors  provides  good  skeletal  muscle  force  estimates. 

Key-Words:  -  sEMG,  ARX,  Weiner-Hammerstein,  KIC,  SI,  GA 


1  Introduction 

Aftereffects  of  the  loss  of  upper  limbs  are  a  reduction 
of  functionality  and  psychological  disturbance  for  the 
person.  According  to  [1]  there  are  1.7  million  peoples 
with  amputation  in  the  United  States  and  this  number  is 
on  rise  after  the  Afghanistan  and  Iraq  war  in  2003  [2]. 
Conversely,  a  prosthetic  limb  can  considerably  increase 
the  functionality  of  an  amputee  and  benefit  the  person  in 
everyday  life. 

In  the  past,  there  have  been  various  research  works 
towards  prosthetic  hand  design,  having  similar 
functionality  and  appearance  as  human  hand  [3-4].  Most 
of  these  research  works  are  based  on  electromyography 
(EMG).  The  EMG  signal  is  activated  and  controlled  by 
the  central  nervous  system,  which  depends  on  the  flow 
of  specific  ions  such  as  sodium  (Aa+),  potassium  (K+) 
and  calcium  (Ca++). 

An  EMG  signal  recorded  on  the  surface  of  the  limb  is 
expressed  as  an  electric  voltage  ranging  between  -5  and 
+5  mV.  This  method  is  known  as  surface 
electromyography  (sEMG).  sEMG  is  utilized  as  an  input 
to  the  controller  to  realize  the  movements  of  the 
prosthesis  and  force  control  [5-6].  Past  research  results 
show  that  EMG  signal  amplitude  generally  increases 
with  skeletal  muscle  force.  However,  this  relationship  is 
not  always  rigid;  various  factors  affect  this  relationship. 


EMG  signals  are  a  result  of  the  varying  motor  unit 
recruitments,  crosstalk,  and  biochemical  interaction 
within  the  muscular  fibres.  This  makes  EMG  signals 
random,  complex  and  dynamic  in  nature  and  the  control 
of  the  prosthesis  difficult.  Moreover,  it  changes 
continuously  due  to  the  onset  and  progression  of  muscle 
fatigue  which  results  because  of  continuous  high 
frequency  stimulation  or  because  of  titanic  stimulation 
[7].  Synchronization  of  active  motor  units  along  the 
muscle  fibres,  and  a  decrease  in  conduction  velocity  are 
reflected  in  the  EMG  signal  as  an  increase  of  amplitude 
in  time  domain  and  a  decrease  of  medium  frequency  in 
frequency  domain  [8].  All  these  factors  make  the 
relationship  between  EMG  and  force  nonlinear.  Correct 
interpretation  of  EMG  signal  is  vital  to  achieve  precise 
motion  and  force  control  of  prosthesis. 

The  present  work  presents  a  novel  approach  to 
estimate  skeletal  muscle  force  using  an  adaptive  multi¬ 
sensor  data  fusion  algorithm  with  hybrid  nonlinear  ARX 
and  Wiener-Hammerstein  models.  Here,  an  array  of 
three  sEMG  sensors  is  used  to  capture  the  information  of 
muscle  dynamics  in  terms  of  sEMG  signals.  The 
recorded  sEMG  signals  are  filtered  utilizing  optimized 
nonlinear  Half-Gaussian  Bayesian  filter  parameters,  and 
the  skeletal  muscle  force  signal  is  filtered  by  using  a 
Chebyshev  type-II  filter.  A  simple  Genetic  Algorithm 
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code  is  used  to  optimize  the  Bayesian  filter  parameters. 
Using  an  input/output  approach,  the  EMG  signal 
measured  at  the  skin  surface  is  considered  as  input  to  the 
skeletal  muscle,  whereas  the  resulting  hand/finger  force 
constitutes  the  output.  Multi  nonlinear  ARX  and 
Wiener-Hammerstein  models  with  different  nonlinearity 
estimators/classes  are  obtained  using  SI  for  three  sets  of 
sensor  data  obtained  from  the  vicinity  of  a  single  motor 
unit.  Different  nonlinearity  estimators/classes  are  used 
for  nonlinear  modeling  as  they  capture  the  dynamics  of 
the  system  differently.  The  outputs  of  estimated 
nonlinear  models  are  fused  with  a  probabilistic  Kullback 
Information  Criterion  (KIC)  for  model  selection  and  an 
adaptive  probability  of  KIC.  First,  the  outputs  are  fused 
for  the  same  sensor  and  for  different  models  and  then  the 
final  outputs  from  each  sensor.  The  final  fused  output  of 
three  sensors  provides  good  skeletal  muscle  force 
estimates. 


Fig.  1:  The  Flow  Chart  for  Skeletal  Muscle  Force  Estimation. 

Fig.  1  shows  the  flow  chart  for  skeletal  muscle  force 
estimation.  This  paper  is  structured  as  follows.  First,  the 
experimental  set-up,  pre-processing  and  filter  parameter 
optimization  for  sEMG  signals  are  discussed.  Second, 
nonlinear  ARX  and  Wiener-Hammerstein  modeling  are 
covered.  Third,  the  fusion  of  various  nonlinear  model  outputs 


using  KIC  and  adaptive  probability  of  KIC  is  covered.  Finally, 
the  results,  discussion  and  future  work  are  provided  followed 
by  a  conclusion  to  summarize  the  importance  of  this  work. 

2  Experimental  Set-Up  and  Pre- 
Processing 

The  experimental  set-up  is  shown  in  Fig.  2.  Both 
sEMG  and  muscle  force  signals  were  acquired 
simultaneously  using  Lab  VIEW™  at  a  sampling  rate  of 
2000  Hz.  The  sEMG  data  capturing  was  aided  by  a 
DELSYS®  Bagnoli-16  EMG  system  with  DE-2.1 
differential  EMG  sensors.  The  corresponding  force  data 
was  captured  using  Interlink  Electronics  FSR  0.5” 
circular  force  sensor.  One  sEMG  sensor  was  placed  on 
the  motor  point  of  the  ring  finger  and  two  adjacent  to 
the  motor  point  of  a  healthy  subject.  Prior  to  placing  the 
sEMG  sensors,  the  skin  surface  of  the  subject  was 
prepared  according  to  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK)  protocols. 


Fig.  2:  Experimental  Set-Up. 

According  to  previous  research,  the  Bayesian  based 
filtering  method  yields  the  most  suitable  sEMG  signals 
[9].  The  nonlinear  filter  significantly  reduces  noise  and 
extracts  a  signal  that  best  describes  EMG  signals  and 
may  permit  effective  use  in  prosthetic  control.  An 
instantaneous  conditional  probability  density  P(EMG\x) 
provides  the  resulting  EMG  for  the  latent  driving  signal 
x  [9].  The  model  for  the  conditional  probability  of  the 
rectified  EMG  signal  emg  =  \EMG\  is  used  in  this 
current  estimation  algorithm.  EMG  signals  are  usually 
described  as  amplitude-modulated  zero  mean  Gaussian 
noise  sequence  [10].  For  the  rectified  EMG  signal,  the 
“Half-Gaussian  measurement  model”  in  [9]  is  given  by 
Equation  (1). 

P(emg \x)  =  2  *  exp(-^y)/(2  *  n  *  x2)1/2.  (1) 

2*xz 

The  EMG  signal  is  modeled  for  the  conditional 
probability  of  the  rectified  EMG  signal  as  a  filtered 
random  process  with  random  rate.  The  likelihood 
function  for  the  rate  evolves  in  time  according  to  a 
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Fokker-Planck  partial  differential  equation  [9].  The 
discrete  time  Fokker-Planck  Equation  is  given  by 
equation  (2). 

p(x,  t— )  «  a  *  p(x  —  a,  t  —  1)  +  (1  —  2  *  a)  *  p(x,  t  — 
1)  +  a  *  p(x  +  £,  £  —  1)  +  /?  +  (1  —  /?)  *  p(x,  £  -  1). 

(2) 

Here,  a  and  (3  are  two  free  parameters,  a  is  the  expected 
rate  of  gradual  drift  in  the  signal,  and  (3  is  the  expected 
rate  of  sudden  shifts  in  the  signal.  The  unknown  driving 
signal  x  is  discretized  into  bins  of  width  £.  These  two 
free  parameters  of  the  non-linear  Half-Gaussian  filter 
model  are  optimized  for  the  acquired  EMG  data  using 
elitism  based  GA. 

A  Chebyshev  type  II  low  pass  filter  with  a  550  Hz 
pass  frequency  is  used  to  filter  the  force  signal.  Fig.  3 
depicts  the  raw  and  Chebyshev  type-II  low  pass  filtered 
force  signals. 


(a)  Raw  Skeletal  Muscle  Force  Signal 


Time  (60.35  secs)  x  io4 

(b)  Chebyshev  Type  -  II  Filtered  Force  Signal 


Fig.  3:  (a)  Raw  and  (b)  Chebyshev  Type-II  Filtered 
Skeletal  Muscle  Force  Signals. 

3  Nonlinear  ARX  and  Wiener- 
Hammerstein  Modeling 

In  this  paper,  we  are  using  nonlinear  ARX  and 
Wiener-Hammerstein  models  with  different  nonlinearity 
estimators/classes  to  model  three  sEMG  sensors  data  as 
input  and  skeletal  muscle  force  data  as  output.  The 
nonlinear  ARX  model  uses  a  parallel  combination  of 
nonlinear  and  linear  blocks  [11]. 

Fig.  4  shows  the  nonlinear  ARX  model  structure.  The 
nonlinear  ARX  model  uses  regressors  as  variables  for 
nonlinear  and  linear  functions.  Regressors  are  functions 
of  measured  input-output  data  [11].  The  predicted  output 
y(t)  of  a  nonlinear  model  at  time  t  is  given  by  the 
general  Equation  (3): 

y(t)  =  F(pc(t))  (3) 

where  x(t)  represents  the  regressors,  F  is  a  nonlinear 
regressor  command,  which  is  estimated  by  nonlinearity 
estimators/classes  [11].  As  shown  in  Fig.  4,  the 
command  F  can  include  both  linear  and  nonlinear 
functions  of  x(t).  Equation  (4)  gives  the  description  of 
F. 

=  Efc=i  ak  K(Pk(x  -  yfc))  (4) 


where  k  is  the  unit  nonlinear  command,  d  is  the  number 
of  nonlinearity  units,  and  ak,  (3k  and  yk  are  the 
parameters  of  the  nonlinearity  estimators/classes  [11]. 


Fig.  4:  Nonlinear  ARX  Model  Structure. 

The  Wiener-Hammerstein  model  uses  one  or  two 
static  nonlinear  blocks  in  series  with  a  linear  block. 
Structural  representation  of  a  nonlinear  Wiener- 
Hammerstein  is  shown  in  Fig.  5  [11]. 


Output  y(t) 

Fig.  5:  Nonlinear  Wiener-Hammerstein  Model  Structure. 

The  general  Equations  (5),  (6),  and  (7)  can  describe 
the  Wiener-Hammerstein  structure  [11]. 
w(t)  =  /(u(t))  (5) 

j:(t)=S)"'(t)  <6) 

y(t)  =  h(x(t)).  (7) 

where  u(t)  and  y(t)  are  input  and  output  of  the  system, 
respectively,  /  and  h  are  nonlinear  functions,  which 
corresponds  to  input  and  output  nonlinearity, 
respectively,  w(t)  and  x(t)  are  internal  variables,  where 
w(t)  has  the  same  dimensions  as  u(t)  and  x(t)  has  the 
same  dimensions  as  y(t),  and  B(q)  and  F(q) 
corresponds  to  the  linear  dynamic  block,  these  are 
polynomials  in  the  backward  shift  operator. 

The  nonlinearity  classes  used  in  this  work  are 
Wavenet,  Treepartition,  Sigmoidnet,  Pwlinear, 
Saturation,  and  Deadzone.  For  motor  point  and  ringl 
sensors,  three  nonlinear  ARX  and  four  nonlinear 
Wiener-Hammerstein  models  with  different  nonlinearity 
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estimators/classes  are  obtained.  For  ring2  sensor,  three 
nonlinear  ARX  and  five  nonlinear  Wiener-Hammer  stein 
models  with  different  nonlinearity  estimators/classes  are 
obtained. 


4  Data  Fusion  and  Adaptive  KIC 
Probability 

Data  fusion  of  multiple  outputs  of  nonlinear  ARX 
and  Wiener-Hammer  stein  models  is  done  by  assigning  a 
particular  probability  to  each  individual  model  [12]. 
First,  the  fusion  algorithm  is  applied  to  the  outputs  of 
different  nonlinear  ARX  and  Wiener-Hammer  stein 
models  for  each  sensor  obtained  using  different 
nonlinearity  estimators.  Second,  the  fusion  algorithm  is 
again  applied  to  the  final  fusion  based  outputs  of  each 
sensor;  this  gives  good  force  estimate.  SI  model  fit 
value  gives  the  probability  for  each  model,  which  is 

[\Y-Y  |1 

1  —  j  *  100.  The  model  selection 

criterion  used  in  this  paper  is  KIC.  The  sum  of  two 
directed  divergences,  which  is  the  measure  of  the 
models  dissimilarity,  is  known  as  Kullback’s  symmetric 
or  J-divergence  [13],  as  given  by  Equation  (8). 

KIC(pi)  =  |  log  Ri  +  ~  nip  (^)  +  g{n),  (8) 

where g(ri)  =  n*  log(n/2). 

The  following  fusion  algorithm  as  given  by  [12]  is 
applied  for  data  fusion  of  the  outputs  of  different 
nonlinear  ARX  and  Wiener-Hammerstein  models: 

1)  Identify  models  Mlf  M2,.-,Mk  using  sEMG  data 
(u)  as  input  and  force  data  (Y)  as  output,  for  k  number 
of  sensors  collecting  data  simultaneously. 

2)  Compute  the  residual  square  norm 

Ri  =  \\Y-<i>iQi\\2  =  \\Y-?\\,  where 
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3)  Calculate  the  model  criteria  coefficient  using 
Equation  (8). 


4)  Compute  the  model  probability  p(M;|Z )  = 


4 


-i/> 
■  e  J 


where  l  is  model  selection  criterion,  i.e.  KIC(pi ). 

5)  Compute  the  fused  model  output 

=  If=iP(W)?;. 

6)  Compute  the  overall  model  from  Yf  and  force  data. 
Here  all  the  computation  from  step  2)  to  6)  is  adaptive 
i.e.  the  residual  square  norm,  KIC(pi ),  model 
probability  p(Mt\Z),  and  fused  model  output  Yf  are 
being  updated  with  time  or  for  each  data  point.  Fig.  6 


shows  the  flow  chart  for  fusion  of  outputs  and  adaptive 
probability  of  KIC. 


Fig.  6:  Data  Fusion  and  Adaptive  KIC  Probability. 


5  Results,  Discussion  and  Future  Work 

This  section  deals  with  the  results,  discussion  and 
future  work.  The  following  plots  show  the  nonlinear 
(ARX  and  Wiener-Hammerstein)  model  and  adaptive 
fusion  algorithm  based  estimated  force  output  for  each 
sensor  first  and  then  finally  combined  adaptive  fusion 
based  output  for  all  three  sensors.  Fig.  7  shows  the 
overlapping  plot  of  the  original  and  adaptive  fusion 
based  force  output  for  the  motor  point  sensor.  The 
output  is  the  result  of  the  adaptive  fusion  algorithm  on 
three  nonlinear  ARX  and  four  nonlinear  Wiener- 
Hammerstein  models  for  the  motor  point  sensor  signal. 


Original  Force  and  NL-ARX  -  NL-HW-2-3-4-5  Models  Fused  Output  -  Sensor  Motor  Point 


Fig.  7:  Original  and  Fusion  Based  Output  for  Motor 
Point  Sensor. 

Fig.  8  shows  the  overlapping  plot  of  the  original  and 
adaptive  fusion  based  force  output  for  ringl  sensor.  The 
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output  is  the  result  of  adaptive  fusion  algorithm  of  three 
nonlinear  ARX  and  four  nonlinear  Wiener-Hammer  stein 
models  for  ringl  sensor  signal. 


Original  Force  and  NL-ARX  -  NL-HW  -  Models  Fused  Force  Output  -  Ringl  Sensor 


Fig.  8:  Original  and  Fusion  Based  Output  for  Ringl 
Sensor. 

Fig.  9  shows  the  overlapping  plot  of  the  original  and 
adaptive  fusion  based  force  output  for  ring2  sensor.  The 
output  is  the  result  of  adaptive  fusion  algorithm  on  three 
nonlinear  ARX  and  five  nonlinear  Wiener-Hammerstein 
models  for  ring2  sensor  signal. 


Original  Force  and  NL-ARX  -  NL-HW  Models  Fused  Output  -  Ring2  Sensor 


Fig.  9:  Original  and  Fusion  Based  Output  for  Ring2 
Sensor. 


Original  Force  and  NL-ARX  -  NL-HW  Models  Output  -  MotorPoint  -  Ringl  -  Ring2  Sensors 


Time  xio4 


Fig.  10:  Final  Plot  -  Original  and  Fusion  Based  Output 
for  All  Three  Sensors. 

Fig.  10  shows  the  overlapping  plot  of  the  original  and 
final  combined  adaptive  fusion  based  force  output  for 


motor  point,  ringl  and  ring2  sensors.  The  output  is  the 
result  of  adaptive  fusion  algorithm  on  the  final  outputs 
of  three  sensors  i.e.  motor  point,  ringl  and  ring2  as 
shown  in  Fig.  7  to  9.  Fig.  10  shows  the  best  skeletal 
muscle  force  estimate,  which  is  the  result  of  the  multi 
nonlinear  ARX  and  Wiener-Hammerstein  models  and  an 
adaptive  hybrid  data  fusion  on  these  nonlinear  models. 
Fig.  11  shows  the  error  plot  of  the  original  and  best- 
estimated  model  output  for  the  motor  point  sensor. 


Error  Plot  -  Original  and  Best  Model  Simulated  Output  for  Motor  Point  Sensor 


Time  x  io4 

Fig.  11:  Error  Plot  -  Original  and  Best-Estimated  Model 
Output  for  Motor  Point  Sensor. 

Fig.  12  shows  the  error  plot  of  original  and  final 
multi  nonlinear  modeled  and  adaptive  hybrid  data  fusion 
based  force  estimate  (results  from  three  sensors, 
nonlinear  modeling  and  adaptive  data  fusion  algorithm). 
If  we  compare  Fig.  11  and  12,  it  is  very  clear  and 
conspicuous  that  the  error  has  decreased  remarkably  and 
is  very  close  to  zero. 


Final  Error  Plot  -  Original  and  Fused  Output  -  MotorPoint,  Ringl  and  Ring2  Sensors 


Fig.  12:  Final  Error  Plot  -  Original  and  Fusion  Based 
Output  for  Motor  Point,  Ringl  and  Ring2  Sensors. 

Future  work  will  focus  on  the  improvement  of  the 
data  collection  techniques  and  experimental  set-up.  By 
using  the  combination  of  linear  and  nonlinear  modeling, 
and  adaptive  hybrid  data  fusion,  the  skeletal  muscle 
force  estimate  can  be  improved  further.  Furthermore,  the 
authors  believe  that  by  using  different  model  selection 
criteria  such  as  Akaike  Information  Criterion  (AIC), 
Kullback  Information  Criterion  (KIC)  and  the  Bayesian 
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Information  Criterion  (BIC)  together  to  obtain  final 
skeletal  muscle  force  estimate  will  give  improved 
results. 

6  Conclusion 

sEMG  and  force  data  acquired  using  three  EMG  and 
one  common  FSR  force  sensor  is  modeled  using 
nonlinear  SI.  Using  different  nonlinearity 
estimators/classes,  multi  nonlinear  ARX  and  Wiener- 
Hammerstein  models  are  obtained  for  each  sensor.  First, 
the  outputs  of  different  models  for  each  sensor  are  fused 
with  a  data  fusion  algorithm  and  an  adaptive  KIC 
probability.  Finally,  the  fused  outputs  from  each  sensor 
are  again  fused  with  same  algorithm  and  adaptive  KIC 
probability.  The  final  estimated  force  using  this 
technique  gives  the  best  estimate. 
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Abstract:  -  Skeletal  muscle  force  and  surface  electromyographic  (sEMG)  signals  are  closely  related.  Hence,  the 
later  can  be  used  for  the  force  estimation.  Usually,  the  location  for  the  sEMG  sensors  is  near  the  respective 
muscle  motor  unit  points.  EMG  signals  generated  by  skeletal  muscles  are  temporal  and  spatially  distributed 
which  results  in  cross  talk  that  is  recorded  by  different  sEMG  sensors.  This  research  focuses  on  modeling 
muscle  dynamics  in  terms  of  sEMG  signals  and  the  generated  muscle  force.  Here,  an  array  of  three  sEMG 
sensors  is  used  to  capture  the  information  of  the  muscle  dynamics  in  terms  of  sEMG  signals  and  generated 
muscle  force.  Optimized  nonlinear  Half-Gaussian  Bayesian  filters  and  a  Chebyshev  type-II  filter  are  used  for 
the  filtration  of  the  sEMG  signals  and  the  muscle  force  signal,  respectively.  A  Genetic  Algorithm  is  used  for  the 
optimization  of  the  filter  parameters.  sEMG  and  skeletal  muscle  force  is  modeled  using  multi  nonlinear  Auto 
Regressive  exogenous  (ARX)  and  Wiener-Hammer  stein  models  with  different  nonlinearity  estimators/classes 
using  System  Identification  (SI)  for  three  sets  of  sensor  data.  An  adaptive  probabilistic  Kullback  Information 
Criterion  (KIC)  for  model  selection  is  applied  to  obtain  the  fusion  based  skeletal  muscle  force  for  each  sensor 
first  and  then  for  the  final  outputs  from  each  sensor.  The  approach  yields  good  skeletal  muscle  force  estimates. 


Key-Words:  -  sEMG,  ARX,  Weiner-Hammer stein,  Prosthetic  hand,  KIC,  System  Identification, 


1  Introduction 

Aftereffects  of  the  loss  of  upper  limbs  are  a 
reduction  of  functionality  and  psychological 
disturbance  for  the  person.  According  to  [1]  there 
are  1.7  million  peoples  with  amputation  in  the 
United  States  and  this  number  is  on  rise  after  the 
Afghanistan  and  Iraq  war  in  2003  [2].  Conversely,  a 
prosthetic  limb  can  considerably  increase  the 
functionality  of  an  amputee  and  benefit  the  person 
in  everyday  life. 

In  the  past,  there  have  been  various  research 
works  towards  prosthetic  hand  design,  having 
similar  functionality  and  appearance  as  human  hand 
[3-4].  Most  of  these  research  works  are  based  on 
electromyography  (EMG).  The  EMG  signal  is 
activated  and  controlled  by  the  central  nervous 
system,  which  depends  on  the  flow  of  specific  ions 
such  as  sodium  ( Na+ ),  potassium  ( K+ )  and  calcium 
(Ca++). 

An  EMG  signal  recorded  on  the  surface  of  the 
limb  is  expressed  as  an  electric  voltage  ranging 
between  -5  and  +5  mV.  This  method  is  known  as 
surface  electromyography  (sEMG).  sEMG  is 


utilized  as  an  input  to  the  controller  to  realize  the 
movements  of  the  prosthesis  and  force  control  [5-6]. 
Past  research  results  show  that  EMG  signal 
amplitude  generally  increases  with  skeletal  muscle 
force.  However,  this  relationship  is  not  always  rigid; 
various  factors  affect  this  relationship.  EMG  signals 
are  a  result  of  the  varying  motor  unit  recruitments, 
crosstalk,  and  biochemical  interaction  within  the 
muscular  fibres.  This  makes  EMG  signals  random, 
complex  and  dynamic  in  nature  and  the  control  of 
the  prosthesis  difficult.  Moreover,  it  changes 
continuously  due  to  the  onset  and  progression  of 
muscle  fatigue  which  results  because  of  continuous 
high  frequency  stimulation  or  because  of  titanic 
stimulation  [7].  Synchronization  of  active  motor 
units  along  the  muscle  fibres,  and  a  decrease  in 
conduction  velocity  are  reflected  in  the  EMG  signal 
as  an  increase  of  amplitude  in  time  domain  and  a 
decrease  of  medium  frequency  in  frequency  domain 
[7].  All  these  factors  make  the  relationship  between 
EMG  and  force  nonlinear.  Correct  interpretation  of 
EMG  signal  is  vital  to  achieve  precise  motion  and 
force  control  of  prosthesis. 
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Fig.  1:  The  Flow  Chart  for  Skeletal  Muscle  Force  Estimation. 


The  present  work  presents  a  novel  approach  to 
estimate  skeletal  muscle  force  using  an  adaptive 
multi-sensor  data  fusion  algorithm  with  hybrid 
nonlinear  ARX  and  Wiener-Hammerstein  models. 
Here,  an  array  of  three  sEMG  sensors  is  used  to 
capture  the  information  of  muscle  dynamics  in 
terms  of  sEMG  signals.  The  recorded  sEMG  signals 
are  filtered  utilizing  optimized  nonlinear  Half- 
Gaussian  Bayesian  filter  parameters,  and  the 
skeletal  muscle  force  signal  is  filtered  by  using  a 
Chebyshev  type-II  filter.  A  simple  Genetic 
Algorithm  code  is  used  to  optimize  the  Bayesian 
filter  parameters.  Using  an  input/output  approach, 
the  EMG  signal  measured  at  the  skin  surface  is 
considered  as  input  to  the  skeletal  muscle,  whereas 
the  resulting  hand/finger  force  constitutes  the 
output.  Multi  nonlinear  ARX  and  Wiener- 
Hammerstein  models  with  different  nonlinearity 


estimators/classes  are  obtained  using  SI  for  three 
sets  of  sensor  data  obtained  from  the  vicinity  of  a 
single  motor  unit.  Different  nonlinearity 
estimators/classes  are  used  for  nonlinear  modeling 
as  they  capture  the  dynamics  of  the  system 
differently.  The  outputs  of  estimated  nonlinear 
models  are  fused  with  a  probabilistic  Kullback 
Information  Criterion  (KIC)  for  model  selection  and 
an  adaptive  probability  of  KIC.  First,  the  outputs  are 
fused  for  the  same  sensor  and  for  different  models 
and  then  the  final  outputs  from  each  sensor.  The 
final  fused  output  of  three  sensors  provides  good 
skeletal  muscle  force  estimates. 

Fig.  1  shows  the  flow  chart  for  skeletal  muscle 
force  estimation.  This  paper  is  structured  as  follows. 
First,  the  experimental  set-up,  pre-processing  and 
filter  parameter  optimization  for  sEMG  signals  are 
discussed.  Second,  nonlinear  ARX  and  Wiener- 
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Fig.  2:  Experimental  Set-Up. 


(a)  Raw  Skeletal  Muscle  Force  Signal 


2  4  6  8  10  12 

Time  (60.35  secs)  x  io4 

(b)  Chebyshev  Type  -  II  Filtered  Force  Signal 


2  4  6  8  10  12 

Time  (60.35  secs)  x  io4 


Fig.  3:  (a)  Raw  and  (b)  Chebyshev  Type-II  Filtered  Skeletal  Muscle  Force  Signals. 


Hammerstein  modeling  is  covered.  Third,  the  fusion 
of  various  nonlinear  model  outputs  using  KIC  and 
adaptive  probability  of  KIC  is  covered.  Finally,  the 
results,  discussion  and  future  work  are  provided 
followed  by  a  conclusion  to  summarize  the 
importance  of  this  work. 


2  Experimental  Set-Up  and  Pre- 
Processing 

The  experimental  set-up  is  shown  in  Fig.  2.  Both 
sEMG  and  muscle  force  signals  were  acquired 
simultaneously  using  Lab  VIEW™  at  a  sampling 
rate  of  2000  Hz.  The  sEMG  data  capturing  was 
aided  by  a  DELSYS®  Bagnoli-16  EMG  system 
with  DE-2.1  differential  EMG  sensors.  The 
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Rectified  EMG  for  Motor  Point  Sensor 


Half  Gaussian  Filtered  EMG  for  Motor  Point  Sensor 


Fig.  4(a):  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Motor  Point  Sensor. 


Rectified  EMG  for  Ring  1  Sensor 


Half  Gaussian  Filtered  EMG  for  Ring  1  Sensor 


Fig.  4(b):  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Ring  1  Sensor. 
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Rectified  EMG  for  Ring  2  Sensor 


Half  Gaussian  Filtered  EMG  for  Ring  2  Sensor 


Fig.  4(c):  Rectified  EMG  and  Half-Gaussian  Filtered  EMG  Signal  for  Ring  2  Sensor. 


corresponding  force  data  was  captured  using 
Interlink  Electronics  FSR  0.5”  circular  force  sensor. 
One  sEMG  sensor  was  placed  on  the  motor  point  of 
the  ring  finger  and  two  adjacent  to  the  motor  point 
of  a  healthy  subject.  Prior  to  placing  the  sEMG 
sensors,  the  skin  surface  of  the  subject  was  prepared 
according  to  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK) 
protocols.  According  to  previous  research,  the 
Bayesian  based  filtering  method  yields  the  most 
suitable  sEMG  signals  [8].  The  nonlinear  filter 
significantly  reduces  noise  and  extracts  a  signal  that 
best  describes  EMG  signals  and  may  permit 
effective  use  in  prosthetic  control.  An  instantaneous 
conditional  probability  density  P(EMG\x)  provides 
the  resulting  EMG  for  the  latent  driving  signal  x  [8]. 
The  model  for  the  conditional  probability  of  the 
rectified  EMG  signal  emg  =  \EMG\  is  used  in  this 
current  estimation  algorithm.  EMG  signals  are 
usually  described  as  amplitude-modulated  zero 
mean  Gaussian  noise  sequence  [9].  For  the  rectified 
EMG  signal,  the  “Half-Gaussian  measurement 
model”  in  [8]  is  given  by  Equation  (1). 

P(emg\x)  =  K  (1) 


The  EMG  signal  is  modeled  for  the  conditional 
probability  of  the  rectified  EMG  signal  as  a  filtered 
random  process  with  random  rate.  The  likelihood 
function  for  the  rate  evolves  in  time  according  to  a 
Fokker-Planck  partial  differential  equation  [8].  The 
discrete  time  Fokker-Planck  Equation  is  given  by 
equation  (2). 

p(x,  t  -)  «  a  *  p(x  —  s,  t  —  1)  +  (1  —  2  *  a)  * 
p(x,  t  —  1)  +  a  *  p(x  +  £,t  —  1)  +  /?  +  (1  —  /?)  * 
p(x,t- 1).  (2) 

Here,  a  and  /?  are  two  free  parameters,  a  is  the 
expected  rate  of  gradual  drift  in  the  signal,  and  /?  is 
the  expected  rate  of  sudden  shifts  in  the  signal.  The 
unknown  driving  signal  x  is  discretized  into  bins  of 
width  s.  These  two  free  parameters  of  the  non-linear 
Half-Gaussian  filter  model  are  optimized  for  the 
acquired  EMG  data  using  elitism  based  GA.  GA 
belongs  to  a  class  of  optimization  algorithms  that 
are  based  on  observing  nature  and  its  corresponding 
processes  to  imitate  solving  complex  problems, 
most  often  optimization  or  estimation  problems,  see 
[10-12].  A  Chebyshev  type  II  low  pass  filter  with  a 
550  Hz  pass  frequency  is  used  to  filter  the  force 
signal.  Fig.  3  depicts  the  raw  and  Chebyshev  type-II 
low  pass  filtered  force  signals.  Fig.  4(a),  (b)  and  (c) 
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show  the  rectified  EMG  and  Half-Gaussian  Filtered 
EMG  signal  for  three  sensors. 


3  Nonlinear  ARX  and  Wiener- 

Hammerstein  Modeling 

In  this  paper,  we  are  using  nonlinear  ARX  and 
Wiener-Hammerstein  models  with  different 
nonlinearity  estimators/classes  to  model  three 
sEMG  sensors  data  as  input  and  skeletal  muscle 
force  data  as  output.  The  nonlinear  ARX  model  uses 
a  parallel  combination  of  nonlinear  and  linear 
blocks  [13]. 

Fig.  5  shows  the  nonlinear  ARX  model  structure. 
The  nonlinear  ARX  model  uses  regressors  as 
variables  for  nonlinear  and  linear  functions. 
Regressors  are  functions  of  measured  input-output 
data  [13].  The  predicted  output  y(t)  of  a  nonlinear 
model  at  time  t  is  given  by  the  general  Equation  (3): 

9(f)  =  F(x(t))  (3) 

where  x(t)  represents  the  regressors,  F  is  a 
nonlinear  regressor  command,  which  is  estimated  by 
nonlinearity  estimators/classes  [13].  As  shown  in 
Fig.  5,  the  command  F  can  include  both  linear  and 
nonlinear  functions  of  x(t).  Equation  (4)  gives  the 
description  of  F. 

F(x)  =  Zk=i  ak  ic(0k(x  -  Yk ))  (4) 

where  k  is  the  unit  nonlinear  command,  d  is  the 
number  of  nonlinearity  units,  and  ak,  j3k  and  yk  are 
the  parameters  of  the  nonlinearity  estimators/classes 
[13]. 

The  Wiener-Hammerstein  model  uses  one  or  two 
static  nonlinear  blocks  in  series  with  a  linear  block. 
Structural  representation  of  a  nonlinear  Wiener- 
Hammerstein  is  shown  in  Fig.  6  [13]. 

The  general  Equations  (5),  (6),  and  (7)  can 
describe  the  Wiener-Hammerstein  structure  [13]. 


w(t )  =  /(u(t )) 

(5) 

* 

ri¬ 

ll 

2^  2?° 
S'  S' 

x 

23 

(6) 

y(t)  =  h(x(t)). 

(7) 

where  u(t)  and  y(t)  are  input  and  output  of  the 
system,  respectively,  /  and  h  are  nonlinear 
functions,  which  corresponds  to  input  and  output 
nonlinearity,  respectively,  w(t)  and  x(t)  are 
internal  variables,  where  w(t)  has  the  same 
dimensions  as  u(t)  and  x(t)  has  the  same 
dimensions  as  y(t),  and  B(q)  and  F(q)  corresponds 
to  the  linear  dynamic  block,  these  are  polynomials 
in  the  backward  shift  operator. 

The  nonlinearity  classes  used  in  this  work  are 
Wavenet,  Treepartition,  Sigmoidnet,  Pwlinear, 
Saturation,  and  Deadzone.  For  motor  point  and 
ringl  sensors,  three  nonlinear  ARX  and  four 


nonlinear  Wiener-Hammerstein  models  with 

different  nonlinearity  estimators/classes  are 

obtained.  For  ring2  sensor,  three  nonlinear  ARX  and 
five  nonlinear  Wiener-Hammerstein  models  with 
different  nonlinearity  estimators/classes  are 

obtained.  Tables  1,  2  and  3  gives  the  nonlinearity 
estimators/classes  for  different  sensors  and  their 
corresponding  model  fit  values. 


Fig.  5:  Nonlinear  ARX  Model  Structure. 


Input  u(t) 


Output  y  (t) 


Fig.  6:  Nonlinear  Wiener-Hammerstein  Model 
Structure. 


4  Data  Fusion  and  Adaptive  KIC 
Probability 

Data  fusion  of  multiple  outputs  of  nonlinear  ARX 
and  Wiener-Hammerstein  models  is  done  by 
assigning  a  particular  probability  to  each  individual 
model  [14].  First,  the  fusion  algorithm  is  applied  to 
the  outputs  of  different  nonlinear  ARX  and  Wiener- 
Hammerstein  models  for  each  sensor  obtained  using 
different  nonlinearity  estimators.  Second,  the  fusion 
algorithm  is  again  applied  to  the  final  fusion  based 
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Table  1:  Different  Nonlinearity  Estimators  and  Model  Fit  Values  for  Motor  Point  Sensor. 


S.  No. 

Model 

Nonlinearity  Class 

Model  Fit 

1. 

nZarxl 

Wavenet 

24.78  % 

2. 

nlarx  2 

Treepartiton 

40.46  % 

3. 

nlarx3 

Sigmoidnet 

48.75  % 

4. 

nlhw2 

Wavenet 

32.38  % 

5. 

nlhw3 

Pwlinear 

26.27  % 

6. 

nlhw4 

Saturation 

55.16% 

7. 

nlhwS 

Deadzone 

42.45  % 

Table  2:  Different  Nonlinearity  Estimators  and  Model  Fit  Values  for  Ringl  Sensor. 


S.  No. 

Model 

Nonlinearity  Class 

Model  Fit 

1. 

nlarx  31 

Wavenet 

-60.76  % 

2. 

nlarx 32 

Treepartiton 

-52.64  % 

3. 

nlarx33 

Sigmoidnet 

-39.36  % 

4. 

nlhw31 

Pwlinear 

31.06% 

5. 

nlhw32 

Sigmoidnet 

10.05  % 

6. 

nlhw33 

Saturation 

31.57% 

7. 

nlhw3S 

Wavenet 

32.36  % 

Table  3:  Different  Nonlinearity  Estimators  and  Model  Fit  Values  for  Ring2  Sensor. 


S.  No. 

Model 

Nonlinearity  Class 

Model  Fit 

1. 

nlarx  21 

Wavenet 

-25.12  % 

2. 

nlarx  22 

Treepartiton 

-22.31  % 

3. 

nlarx  23 

Sigmoidnet 

-33.65  % 

4. 

nlhw21 

Pwlinear 

34.69  % 

5. 

nlhw22 

Sigmoidnet 

34.76  % 

6. 

nlhw23 

Saturation 

33.3  % 

7. 

nlhw2A 

Deadzone 

34.94  % 

8. 

nlhw2S 

Wavenet 

34.89  % 

outputs  of  each  sensor;  this  gives  good  force 
estimate.  SI  model  fit  value  gives  the  probability  for 

[\Y— Y  |1 
1  —  ^  *  100. 

The  model  selection  criterion  used  in  this  paper  is 
KIC.  The  sum  of  two  directed  divergences,  which  is 
the  measure  of  the  models  dissimilarity,  is  known  as 
Kullback’s  symmetric  or  J-divergence  [15],  as  given 
by  Equation  (8). 


,  n  (pi  +  l)n 

^/c(Pi)=-iogfii+yt  % 

2  n  —  pi  —  Z 


nip  (!L^i)  +  g(n), 


(8) 


where  g(n )  =  n  *  log(n/2). 


The  following  fusion  algorithm  as  given  by  [14]  is 
applied  for  data  fusion  of  the  outputs  of  different 
nonlinear  ARX  and  Wiener-Hammer  stein  models: 

1)  Identify  models  Mlt  using  sEMG 

data  ( u )  as  input  and  force  data  (Y)  as  output, 
for  k  number  of  sensors  collecting  data 
simultaneously. 

2)  Compute  the  residual  square  norm 


Y  — 

*aii2 = 

\\y- 

n> 
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Fig.  7:  Data  Fusion  and  Adaptive  KIC  Probability. 


3)  Calculate  the  model  criteria  coefficient  using 
Equation  (8). 

4)  Compute  the  model  probability  p(MjZ)  = 

e-h 

— — r/-,  where  l  is  model  selection  criterion, 

lj=1e  V 

i.e.  KIC(pi ). 

5)  Compute  the  fused  model  output  Yf  = 

zUpmztfi. 

6)  Compute  the  overall  model  from  Yf  and  force 
data. 

Here  all  the  computation  from  step  2)  to  6)  is 
adaptive  i.e.  the  residual  square  norm,  KIC(pi ), 
model  probability  p(M;|Z),  and  fused  model  output 
Yf  are  being  updated  with  time  or  for  each  data 
point.  Fig.  7  shows  the  flow  chart  for  fusion  of 
outputs  and  adaptive  probability  of  KIC. 


5  Results,  Discussion  and  Future 
Work 

This  section  deals  with  the  results,  discussion  and 
future  work.  The  following  plots  show  the  nonlinear 


(ARX  and  Wiener-Hammerstein)  model  and 
adaptive  fusion  algorithm  based  estimated  force 
output  for  each  sensor  first  and  then  finally 
combined  adaptive  fusion  based  output  for  all  three 
sensors.  Fig.  8  shows  the  overlapping  plot  of  the 
original  and  adaptive  fusion  based  force  output  for 
the  motor  point  sensor.  The  output  is  the  result  of 
the  adaptive  fusion  algorithm  on  three  nonlinear 
ARX  and  four  nonlinear  Wiener-Hammerstein 
models  for  the  motor  point  sensor  signal.  Fig.  9 
shows  the  overlapping  plot  of  the  original  and 
adaptive  fusion  based  force  output  for  ringl  sensor. 
This  output  is  the  result  of  adaptive  fusion  algorithm 
of  three  nonlinear  ARX  and  four  nonlinear  Wiener- 
Hammerstein  models  for  ringl  sensor  signal.  Fig.  10 
shows  the  overlapping  plot  of  the  original  and 
adaptive  fusion  based  force  output  for  ring2  sensor. 
This  output  is  the  result  of  adaptive  fusion  algorithm 
on  three  nonlinear  ARX  and  five  nonlinear  Wiener- 
Hammerstein  models  for  ring2  sensor  signal.  Fig.  11 
shows  the  overlapping  plot  of  the  original  and  final 
combined  adaptive  fusion  based  force  output  for 
motor  point,  ringl  and  ring2  sensors.  The  output  is 
the  result  of  adaptive  fusion  algorithm  on  the  final 
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Original  Force  and  NL-ARX  -  NL-HW-2-3-4-5  Models  Fused  Output  -  Sensor  Motor  Point 


Time  xio4 

Fig.  8:  Original  and  Fusion  Based  Output  for  Motor  Point  Sensor. 


Original  Force  and  NL-ARX  -  NL-HW  -  Models  Fused  Force  Output  -  Ringl  Sensor 


Fig.  9:  Original  and  Fusion  Based  Output  for  Ringl  Sensor. 
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Original  Force  and  NL-ARX  -  NL-HW  Models  Fused  Output  -  Ring2  Sensor 


Time  xio4 

Fig.  10:  Original  and  Fusion  Based  Output  for  Ring2  Sensor. 


Original  Force  and  NL-ARX  -  NL-HW  Models  Output  -  MotorPoint  -  Ringl  -  Ring2  Sensors 


-/ 


mm* 


s5-5  Hr 


0.4 


, :  !  Ur, 

i' ti 

“HI  II  ;H 


°-2! “ii  i 

'  I 


0.1 


TTfij 


SL 


!HL 


h- 


4. 

m: 


!  Ill 


'  iiiiii1 

jj  Hi 
liil  r i 
'!  i 


lii'n 

Hifci 


-Nf'; 


i  I 


Hit 


ijllii, 


’''iflnlilHiff  + 


>  id,  ii 


,  ii 

Ii  ! 


1 

0 

4 

ft  I  ill  i'hjf 

4 

' 

ii 


)t  lij 

-  L  J I J  I 


It 


Time  xio4 

Fig.  1 1 :  Final  Plot  -  Original  and  Fusion  Based  Output  for  All  Three  Sensors. 
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A  mplitude 


Error  Plot  ■  Original  and  Best  Model  Simulated  Output  for  Motor  Point  Sensor 


Time 

Fig.  12:  Error  Plot  -  Original  and  Best-Estimated  Model  Output  for  Motor  Point  Sensor. 


FinalError  Plot  ■  Original  andFused  Output  -MotorPointRingl  andRing2  Sensors 


Fig.  13:  Final  Error  Plot  -  Original  and  Fusion  Based  Output  for  Motor  Point,  Ringl  and  Ring2  Sensors. 
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outputs  of  three  sensors  i.e.  motor  point,  ringl  and 
ring2  as  shown  in  Fig.  8  to  10.  Fig.  11  shows  the 
best  skeletal  muscle  force  estimate,  which  is  the 
result  of  the  multi  nonlinear  ARX  and  Wiener- 
Hammerstein  models  and  an  adaptive  hybrid  data 
fusion  on  these  nonlinear  models.  Fig.  12  shows  the 
error  plot  of  the  original  and  best-estimated  model 
output  for  the  motor  point  sensor. 

Fig.  13  shows  the  error  plot  of  original  and  final 
multi  nonlinear  modeled  and  adaptive  hybrid  data 
fusion  based  force  estimate  (results  from  three 
sensors,  nonlinear  modeling  and  adaptive  data 
fusion  algorithm).  If  we  compare  Fig.  12  and  13,  it 
is  very  clear  and  conspicuous  that  the  error  has 
decreased  remarkably  and  is  very  close  to  zero. 

Future  work  will  focus  on  the  improvement  of 
the  data  collection  techniques  and  experimental  set¬ 
up.  By  using  the  combination  of  linear  and 
nonlinear  modeling,  and  adaptive  hybrid  data 
fusion,  the  skeletal  muscle  force  estimate  can  be 
improved  further.  Furthermore,  the  authors  believe 
that  by  using  different  model  selection  criteria  such 
as  Akaike  Information  Criterion  (AIC),  Kullback 
Information  Criterion  (KIC)  and  the  Bayesian 
Information  Criterion  (BIC)  together  to  obtain  final 
skeletal  muscle  force  estimate  will  give  improved 
results. 


6  Conclusions 

sEMG  and  force  data  acquired  using  three  EMG  and 
one  common  FSR  force  sensor  is  modeled  using 
nonlinear  SI.  Using  different  nonlinearity 
estimators/classes,  multi  nonlinear  ARX  and 
Wiener-Hammerstein  models  are  obtained  for  each 
sensor.  First,  the  outputs  of  different  models  for 
each  sensor  are  fused  with  a  data  fusion  algorithm 
and  an  adaptive  KIC  probability.  Finally,  the  fused 
outputs  from  each  sensor  are  again  fused  with  same 
algorithm  and  adaptive  KIC  probability.  The  final 
estimated  force  using  this  technique  gives  the  best 
estimate.  The  presented  approach  can  be  utilized  for 
controlling  prosthetic  hands  [16]. 
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Abstract  -  Position  and  force  control  are  two  critical  aspects 
of  prosthetic  control.  Surface  electromyographic  (sEMG) 
signals  can  be  used  for  skeletal  muscle  force  estimation.  In 
this  paper,  skeletal  muscle  is  considered  as  a  system  and 
System  Identification  (SI)  is  used  to  model  sEMG  and  skeletal 
muscle  force.  The  recorded  sEMG  signal  is  filtered  utilizing 
optimized  nonlinear  Half-Gaussian  Bayesian  filter,  and  a 
Chebyshev  type-II  filter  prepares  the  muscle  force  signal.  The 
filter  optimization  is  accomplished  using  Genetic  Algorithm 
(GA).  Multi-  linear  and  nonlinear  models  are  obtained  with 
sEMG  as  input  and  skeletal  muscle  force  of  a  human  hand  as 
an  output.  The  outputs  of  these  models  are  fused  with  a 
probabilistic  Kullback  Information  Criterion  (KIC)  for  model 
selection  and  an  adaptive  probability  of  KIC.  This  approach 
gives  good  estimate  of  the  skeletal  muscle  force. 


I.  Introduction 

The  current  incidence  of  amputation  in  the  United 
States  is  approximately  1.7  million  people  [1]  and  this 
number  is  on  rise  with  the  U.S.  involvement  in  both  the 
Afghanistan  and  Iraq  wars  since  2003  [2].  Loss  of  upper 
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limb  results  in  reduction  of  functionality  and  presents 
numerous  psychological  issues  for  the  individual. 

Previous  research  was  mainly  based  on  the 
electromyography  (EMG)  signals  and  focus  on  the 
prosthetic  hand  design  [3-4].  The  EMG  signal  recorded  at 
the  surface  of  the  limb  is  known  as  surface  EMG  (sEMG) 
which  is  an  electric  voltage  ranging  between  -5  and  +5 
mV.  sEMG  can  be  used  to  control  the  position  and  force  of 
the  hand  prosthesis  [5-6].  Generally  EMG  signal  amplitude 
increases  with  an  increase  in  the  skeletal  muscle  force, 
however,  various  factors  affect  this  relationship.  Since 
varying  motor  unit  recruitments,  crosstalk,  and 
biochemical  interaction  within  the  muscular  fibres 
contribute  to  generate  EMG  signals,  hence  the  EMG 
signals  are  random,  complex  and  dynamic  in  nature. 
Together  all  of  these  factors  make  the  relationship  between 
EMG  and  finger  force  nonlinear.  Precise  control  of  the 
prosthesis  is  critical,  so  the  correct  interpretation  of  the 
measured  EMG  signal  is  important.  Previously  authors 
have  published  similar  work  using  multi  sensor  data  fusion 
and  Output  Error  (OE)  model  and  modal  transformation 

[7] ,  and  multi  sensor  data  fusion  using  nonlinear  models 

[8] . 

The  present  work  presents  a  novel  approach  to  estimate 
skeletal  muscle  force  using  an  adaptive  data  fusion 
algorithm  with  hybrid  multi  linear  and  nonlinear  models  as 
shown  in  Figure  1.  The  recorded  sEMG  signals  are  filtered 
utilizing  Genetic  Algorithm  (GA)  based  optimized 
nonlinear  Half-Gaussian  Bayesian  filter  parameters,  and 
the  skeletal  muscle  force  signal  is  filtered  by  using  a 
Chebyshev  type-II  filter.  Using  an  input/output  approach, 
the  EMG  signal  measured  at  the  skin  surface  is  considered 
as  input  to  the  skeletal  muscle,  whereas  the  resulting 
hand/fmger  force  constitutes  the  output.  Multi  linear  and 
nonlinear  models  are  obtained  using  System  Identification 
(SI)  for  the  motor  point  signals  as  they  capture  the 
dynamics  of  the  system  differently.  The  outputs  of 
estimated  linear  and  nonlinear  models  are  fused  with  a 
probabilistic  Kullback  Information  Criterion  (KIC)  for 
model  selection  and  an  adaptive  probability  of  KIC.  First, 
the  outputs  are  fused  for  the  linear  models  and  then  for 
nonlinear  models  and  finally  the  resultant  outputs  of  linear 
and  nonlinear  models  are  fused  together.  The  final  fused 
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output  of  different  linear  and  nonlinear  models  results  in 
good  skeletal  muscle  force  estimates. 


Fig.  1.  Flow  chart  for  estimation  of  skeletal  muscle  force 
using  an  adaptive  data  fusion  algorithm. 

II.  Experimental  set-up  and  pre-processing 
Data  acquisition  for  EMG  and  skeletal  muscle  force  is 
accomplished  using  Lab  VIEW™  8.2  simultaneously  at  a 
sampling  rate  of  2000  Hz.  A  DELSYS®  Bagnoli-16  EMG 
system  with  DE-2.1  differential  EMG  sensors  is  used  for 
sEMG  data  capturing.  However,  the  corresponding  force 
data  was  captured  using  NI  ELVIS  with  Interlink 
Electronics  FSR  0.5”  circular  force  sensor.  Figure  2 
explains  the  experimental  set-up.  One  sEMG  sensor  was 
placed  on  the  motor  point  of  the  ring  finger  and  two 
adjacent  to  the  motor  point  of  a  healthy  subject.  For  the 
experiment,  three  sEMG  sensors  were  used,  but  only  the 
motor  point  signal  is  used  for  the  force  estimation  in  this 
research  work.  The  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK)  protocol  was 
followed  to  capture  the  sEMG  signals. 


Fig.  2.  Experimental  set-up. 

To  model  the  sEMG  signal,  it  is  assumed  that  the 
conditional  probability  of  the  rectified  EMG  signal  is  a 
filtered  random  process  with  random  rate.  Fokker-Planck 


partial  differential  equation  gives  the  likelihood  function 
for  the  rate  evolving  in  time  [9].  In  the  Fokker-Planck 
partial  differential  equation,  a  and  /?  are  two  free 
parameters,  a  is  the  expected  rate  of  gradual  drift  in  the 
signal,  and  /?  is  the  expected  rate  of  sudden  shifts  in  the 
signal.  Optimization  of  these  two  free  parameters  is 
achieved  using  the  GA.  Skeletal  muscle  force  signal  is 
filtered  with  a  Chebyshev  type  II  low  pass  filter. 

III.  System  identification  -  linear  and  non¬ 
linear  MODELING 

System  Identification  (SI)  is  based  on  the  acquired 
experimental  data  and  is  an  alternative  to  physically  based 
mathematical  modeling  [10].  A  dynamical  model  such  as 
the  Auto  Regressive  with  exogenous  input  (ARX)  or 
Output  Error  (OE)  gives  the  system  in  mathematical  form 
which  is  the  outcome  of  parametric  system  identification 
process.  In  this  work,  the  SI  approach  is  used  with  the 
myolelectric  signal  as  the  input  to  the  system  and  as  the 
output  is  the  intended  fmger/hand  force.  In  this  work, 
mutli-  linear  and  nonlinear  models  are  obtained  for 
modeling  of  EMG  and  hand/fmger  force  signals.  Six  linear 
and  three  nonlinear  models  are  obtained  for  the  motor 
point  sEMG  signal. 


TABLE  I 

LINEAR  MODELS  AND  THEIR  STRUCTURES 


Linear  Model  Name 

Linear  Model  Structure 

Output  Error 

y(0  =j^u^~nk)  +  e{t) 

ARX 

A(q)y(t)  =  B(q)u(t  -  nk)  +  e(t) 

ARM  AX 

A(q)y(t)  =  B(q)u(t-nk ) 

+  C(q)e(t) 

Box-Jenkins 

yit)=W)uit~nk)+me(t) 

State-Space  -  subspace 
method 

x(t  +  Ts)  =  Ax(t)  +  Bu(t)  +  Ke(t) 

y(t)  =  Cx{t)  +  Du(t)  +  e(t) 

State-Space  -  prediction 
error/maximum 
likelihood  method 

x(t  +  Ts)  =  Ax(t)  +  Bu{t)  +  Ke(t) 

y(t)  =  Cx(t)  +  Du(t)  +  e(t) 

For  the  linear  models  Output  Error  model  of  order  6, 
ARX  model  of  order  6,  ARM  AX  model  of  model  order  5, 
Box-Jenkins  model  of  order  18,  State-Space  model  with 
subspace  method  of  order  12  and  a  State-Space  model  with 
prediction  error/maximum  likelihood  method  of  order  12 
are  obtained  using  SI  as  given  in  table  I.  In  table  I  y  is 
output,  t  is  time,  B(q),  F(q),  A(q),  C(q),  D(q),  are 
polynomials,  q  is  a  backward  shift  operator,  u  is  input,  nk 
is  delay  and  e  is  error  [11].  For  State- Space  models:-  x  is 
state,  t  is  time,  Ts  is  sampling  time,  u  is  input,  e  is  error, 
A,  B ,  K,  C,  and  D  are  system  matrices,  and  y  is  output  [11]. 
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In  contrary  to  six  linear  models  as  given  in  table  I, 
nonlinear  ARX  with  nonlinearity  estimator  as  ‘sigmoidnet’ 
and  nonlinear  Wiener-Hammerstein  models  with 
nonlinearity  estimators  as  ‘piecewise  linear  -  pwlinear’  and 
‘saturation’  are  obtained,  as  given  in  table  II. 


TABLE  II 

NONLINEAR  MODELS  AND  THEIR  STRUCTURES 


Nonlinear  Model  Name 

Nonlinear  Model  Structure 

Nonlinear  ARX  - 
Sigmoidnet 

yP(0  =  l),y(t- 

2 ),y(t  —  3),  —  1  ),u(t  — 

2),...); 

f(z)  =  is  sigmoid 

function. 

Nonlinear  HW  -  Piecewise 
Linear  -pwlinear 

w(t)  =  /(u(t)), 

*(0  =  ^w(t), 

F]M) 

y(t)  =  h(x(t )); 

Nonlinear  HW  -  Saturation 

II 

IT 

*(0  =  77?N(t), 

F]M) 

II 

✓ - N 

In  table  II  /  is  a  nonlinear  function  for  the  nonlinear 
ARX  model.  Inputs  to  /  are  model  regressors.  yv(t)  is  the 
predicted  output  as  a  weighted  sum  of  past  output  values 
and  current  and  past  input  values.  For  the  nonlinear 
Hammerstein- Wiener  model  u(t)  and  y(t)  are  the  inputs 
and  outputs  for  the  system,  respectively.  /  and  h  are 
nonlinear  functions  that  corresponds  to  the  input  and 
output  nonlinearity,  respectively.  w(t)  and  x(t)  are 
internal  variables.  w(t)  has  the  same  dimension  as  u{t). 
x(t)  has  the  same  dimension  as  y(t).  B(q)  and  F(q)  in  the 
linear  dynamic  block  are  polynomials  in  the  backward  shift 
operator.  If  only  the  input  nonlinearity  is  present,  the 
model  is  called  the  Hammerstein  model.  If  only  the  output 
nonlinearity  is  present,  the  model  is  called  the  Wiener 
model  [11]. 

IV.  Fusion  of  outputs  and  adaptive  probability 

of  KIC 

The  outputs  of  multiple  models  can  be  fused  by 
assigning  a  particular  probability  to  each  individual  model 
[12].  In  this  work,  we  are  obtaining  multi  linear  and 
nonlinear  models  using  SI  for  the  sEMG  sensor  for  the 
motor  unit.  The  probability  of  each  model  is  computed 
using  the  SI  model  fit  value.  Model  fit  value  is  given  by 

[l  —  *  100.  Kullback  information  criterion  (KIC), 

which  is  an  asymmetric  measure  for  the  model  selection,  is 


used  for  this  research.  Kullback’s  symmetric  or  J- 
divergence  is  the  sum  of  two  directed  divergences,  which  is 
the  measure  of  the  models  dissimilarity  and  given  by 
Equation  (1)  [13]. 

KIC(Pi)  =  ~ log  ~  ™p  Cjp)  +  g(n),  (1) 

where  g(n)  =  n  *  log(n/2). 

Based  on  [10],  the  following  fusion  technique  is 
applied  for  sEMG  -  force  identification  models. 

1)  Identify  OE  models  Mlf  M2,  using  sEMG  data 

( u )  as  input  and  force  data  (T)  as  output,  for  k  1  number  of 
sensors  collecting  data  simultaneously. 

2)  Compute  the  residual  square  norm 

Rt  =  ||r  -  Of©i||2  =  ||r-  f||2, 

where  and 
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3)  Calculate  the  model  criteria  coefficient  using  Equation 

(1). 

e~li 

4)  Compute  the  model  probability  p(M;|Z)  =  — — — , 

£;= ie  1 

where  l  is  model  selection  criterion,  i.e.  K/C(pj). 

5)  Compute  the  fused  model  output  Yf  =  Yli=i  p(Mj|Z)fj. 

6)  Compute  the  overall  OE  model  from  Yf  and  force  data. 
In  the  above  steps  all  the  computation  from  step  2)  to  step 
6)  is  adaptive  i.e.  the  residual  square  norm,  K/C(pj), 
model  probability  p(M;|Z),  and  fused  model  output  Yf  are 
being  updated  with  time  or  for  each  data  point. 

V.  Results,  conclusion  and  future  work 
Adaptive  KIC  probability  based  data  fusion  algorithm 
is  applied  to  linear  and  nonlinear  models  separately  and 
then  combined.  Figures  3  and  4  show  the  results  for  the 
linear  and  nonlinear  models  respectively.  Finally,  both 
linear  and  nonlinear  models  are  combined  and  adaptive 
KIC  probability  based  data  fusion  algorithm  is  applied  to 
obtain  the  final  output  that  is  given  by  Figure  5.  Validation 
of  this  technique  is  confirmed  with  a  separate  data  set. 
Figure  6  depicts  the  validation  of  the  proposed  technique. 
Future  work  will  focus  on  the  improved  data  collection 
techniques,  use  of  spatial  filters  for  sEMG  signals 
filtration,  and  use  of  multi  sensor  data  with  this  technique. 


KIC  Adaptive  Probability  -  Data  Fusion  Based  Force -Linear  Models  and  Chebyshev  Filtered  Force  Signal 


Time  (60  Seconds  Data) 


Fig.  3.  For  linear  models:  KIC  adaptive  probability  data 
fusion  based  force  vs.  Chebyshev  type  II  filtered  force. 
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Time  (60  Seconds  Data)  *io4 


Fig.  4.  For  nonlinear  models:  KIC  adaptive  probability  data 
fusion  based  force  vs.  Chebyshev  type  II  filtered  force. 


Time  (60.32  Seconds  Data)  xio4 


Fig.  5.  For  linear  and  nonlinear  models  combined:  KIC 
adaptive  probability  data  fusion  based  force  vs.  Chebyshev 
type  II  filtered  force. 


Validation  Plot  -  Measured  Output  Vs.  Estimated  Fused  Output  With  Adaptive  Probability 


Time  (37.36  secs) 


Fig.  6.  Validation  plot  -  for  linear  and  nonlinear  models 
combined:  KIC  adaptive  probability  data  fusion  based 
force  vs.  Chebyshev  type  II  filtered  force. 
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Abstract  -  Effective  use  of  upper  extremity  prostheses  depends  on 
the  two  critical  aspects  of  precise  position  and  force  control. 
Surface  electromyographic  (sEMG)  signals  can  be  used  as  a 
control  input  for  the  position  and  force  actions  related  to  the 
prosthesis.  In  this  paper ,  we  use  the  measured  sEMG  signals  to 
estimate  skeletal  muscle  force.  Further ,  we  consider  skeletal  muscle 
as  a  system  and  System  Identification  (SI)  is  used  to  model  multi¬ 
sensor  sEMG  and  skeletal  muscle  force.  The  sEMG  signals  are 
filtered  utilizing  optimized  nonlinear  Half-Gaussian  Bayesian  filter , 
and  a  Chebyshev  type-II  filter  provides  the  muscle  force  signal.  The 
filter  optimization  is  accomplished  using  a  Genetic  Algorithm  (GA). 
Multi-  linear  and  nonlinear  models  are  obtained  with  sEMG  data 
as  input  and  skeletal  muscle  force  of  a  healthy  human  hand  as  an 
output  for  three  sensors.  The  outputs  of  these  models  for  three 
sensors  are  fused  with  a  probabilistic  Kullback  Information 
Criterion  (KIC)  for  model  selection  and  an  adaptive  probability  of 
KIC.  The  final  fusion  based  force  for  multi-sensor  sEMG  gives 
improved  estimate  of  the  skeletal  muscle  force. 

Keywords:  sEMG,  SI,  GA,  KIC 

I.  Introduction 

In  the  last  decade  there  have  been  active  research  efforts 
in  the  field  of  prosthetics.  According  to  [1]  the  United  States 
has  1.7  million  of  amputees  and  this  figure  is  on  the  rise  since 
2003  due  to  the  wars  in  Afghanistan  and  Iraq  [2]. 
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Individuals  with  upper  extremity  amputations  not  only  have 
a  reduction  in  function,  but  also  struggle  with  numerous 
psychological  issues  which  may  further  complicate  the 
appropriate  control  and  use  of  the  prosthesis.  Previous 
research  on  the  prosthetic  hand  design  was  mainly  based  on 
the  electromyography  (EMG)  signals  [3-4].  The  EMG  signal 
recorded  at  the  surface  of  the  limb  is  known  as  surface  EMG 
(sEMG).  The  sEMG  is  an  electric  voltage  ranging  between  -5 
and  +5  mV,  which  reflects  the  electrical  activity  associated 
with  voluntary  muscle  contraction  and  which  has  been  used  to 
control  the  position  and  force  of  the  hand  prosthesis  [5-6]. 
The  skeletal  muscle  force  and  the  sEMG  signals  are  related 
and  higher  sEMG  amplitude  generally  correlates  with  greater 
force  production.  However,  there  are  various  other  factors 
that  affect  this  relationship.  The  sEMG  signals  are  random, 
complex  and  dynamic  in  nature  because  of  varying  motor  unit 
recruitment,  crosstalk,  and  biochemical  interaction  within  the 
muscular  fibers.  Hence  all  these  factors  contribute  towards 
the  nonlinear  relationship  between  the  two. 

Since  sEMG  is  the  control  input  for  the  prosthesis,  precise 
control  and  correct  interpretation  of  the  measured  sEMG 
signal  is  important.  In  previous  works,  the  authors  have 
published  similar  work  using  multi  sensor  data  fusion  and 
Output  Error  (OE)  model  and  modal  transformation  [7],  and 
multi  sensor  data  fusion  using  nonlinear  models  [8]. 

The  present  work  is  an  extension  of  our  previous  work  [9] 
where  we  proposed  estimation  of  the  skeletal  muscle  force 
using  an  adaptive  data  fusion  algorithm  with  hybrid  multi 
linear  and  nonlinear  models.  In  [9]  the  sEMG  data  was  based 
only  from  measuring  the  motor  point  sEMG  signal,  whereas 
in  the  present  work  we  extend  this  to  three  sensors  for  the 
ring  finger  of  the  dominant  hand  of  a  healthy  male  subject. 
Figure  1  presents  the  flow  chart  of  the  work  done  in  this 
paper.  The  sEMG  signals  are  acquired  from  the  forearm  of  a 
healthy  subject  and  filtered  utilizing  Genetic  Algorithm  (GA) 
based  optimized  nonlinear  Half-Gaussian  Bayesian  filter.  On 
the  other  hand,  the  skeletal  muscle  force  signal  is  filtered 
using  a  Chebyshev  type-II  filter.  The  sEMG  is  considered  as 
input  whereas  force  signal  is  considered  as  an  output.  Since 
each  model  captures  the  dynamics  differently,  we  obtained 
multiple  linear  and  nonlinear  models  using  System 
Identification  (SI)  for  the  sEMG  data  corresponding  to  the 
three  sensors.  First,  the  outputs  of  all  the  linear  and  nonlinear 
models  from  the  three  sensors  are  fused  separately  and  then 
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the  resultant  three  outputs  are  fused  with  a  probabilistic 
Kullback  Information  Criterion  (KIC)  for  model  selection  and 
an  adaptive  probability  of  KIC.  The  final  fused  output  using 
this  approach  gives  improved  skeletal  muscle  force  estimates. 

The  paper  is  organized  as  follows,  the  present  section 
covers  the  introduction  and  literature  work,  which  is  followed 
by  the  description  of  the  experimental  set-up  and  pre¬ 
processing,  then  system  identification  and  linear  and 
nonlinear  modeling  is  discussed,  followed  by  the  data  fusion 
algorithm.  The  paper  is  concluded  by  a  section  with,  results 
and  discussions,  and  followed  by  the  conclusions  and  future 
work. 


Figure  1.  Flow  chart  for  estimation  of  skeletal  muscle  force 
using  an  adaptive  data  fusion  algorithm. 

II.  Experimental  Set-Up  and  Pre-Processing 

The  sEMG  and  skeletal  muscle  force  signals  are  acquired 
using  Lab  VIEW™  8.2  simultaneously  at  a  sampling  rate  of 
2000  Hz.  A  DELSYS®  Bagnoli-16  EMG  system  with  DE-2.1 
differential  EMG  sensors  is  used  for  sEMG  data  capturing. 
However,  the  corresponding  force  data  was  captured  using  NI 
ELVIS  with  Interlink  Electronics  FSR  0.5”  circular  force 
sensor.  The  experimental  set-up  is  shown  in  Figure  2.  One 


sEMG  sensor  was  placed  on  the  motor  point  of  the  ring  finger 
and  two  adjacent  to  the  motor  point  of  a  healthy  subject.  Skin 
surface  on  the  forearm  of  the  subject  was  prepared  with  the 
International  Society  of  Electrophysiology  and  Kinesiology 
(ISEK)  protocol  prior  to  placing  sEMG  sensors. 

The  nonlinear  Bayesian  filter  yields  good  sEMG  data  and 
significantly  reduces  the  influence  of  noise  and  it  also  extracts 
a  signal  that  best  describes  the  EMG  signals  and  may  permit 
effective  use  in  prosthetic  control  [10].  The  sEMG  signal  can 
be  modeled  with  the  assumption  that  the  conditional 
probability  of  the  rectified  EMG  signal  is  a  filtered  random 
process  with  random  rate.  The  “Half-Gaussian  measurement 
model”  for  the  rectified  EMG  signal  is  given  by  Equation  (1), 
[10]. 


Figure  2.  Experimental  Set-Up. 


P(emg\x )  =  2  *  exp(— ^%-)/( 2  *  n  *  x2)1/2.  (1) 

The  likelihood  function  for  the  rate  evolving  in  time  is  given 
by  the  Fokker-Planck  partial  differential  equation,  where  the 
two  free  parameters  in  the  signal  are  the  expected  rate  of 
gradual  drift,  a  and  the  expected  rate  of  sudden  shifts  /?  [10]. 
The  two  free  parameters  of  the  sEMG  signal  are  optimized 
using  an  elitism  based  GA.  On  the  other  hand  the  skeletal 
muscle  force  signal  is  filtered  with  a  Chebyshev  type  II  low 
pass  filter. 

III.  System  Identification  -  Linear  and  Nonlinear 
Modeling 

A  system  can  be  modeled  either  by  using  physics  based 
mathematical  models  or  by  System  Identification  (SI)  [11]. 
Parametric  SI  gives  a  dynamic  model  such  as  the  Auto 
Regressive  with  exogenous  input  (ARX)  or  Output  Error 
(OE)  of  the  system  in  mathematical  form.  In  this  research,  we 
use  the  SI  approach  to  model  the  two  signals.  The  sEMG  is 
the  input  to  the  system  and  the  intended  fmger/hand  force  is 
output.  In  this  work,  mutli-  linear  and  nonlinear  models  are 
obtained  for  modeling  of  sEMG  and  hand/fmger  force  signals 
for  three  sensors  for  the  ring  finger  of  the  dominant  hand  of  a 
healthy  subject.  Six  linear  and  three  nonlinear  models  are 
obtained  for  the  motor  point  sEMG  signal,  three  linear  and 
five  nonlinear  models  are  obtained  for  the  ringl  sensor,  and 
six  linear  and  five  nonlinear  models  are  obtained  for  ring2 
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sensor  data.  Table  I  and  II  gives  the  structures  of  linear  and 
nonlinear  models  respectively. 

The  model  order  of  the  various  models  used  in  this  work 
are  as  follows:  linear  models  for  the  motor  point  data,  OE 
model  of  order  6,  ARX  model  of  order  6,  ARMAX  model  of 
model  order  5,  Box-Jenkins  (BJ)  model  of  order  18,  State- 
Space  model  with  subspace  method  (N4SID)  of  order  12  and 
a  State- Space  model  with  prediction  error/maximum 
likelihood  method  (PEM)  of  order  12  are  obtained  using  SI. 
For  ringl  sensor  (sensor  adjacent  to  the  motor  point  sensor) 
data  OE  model  of  order  17,  ARMAX  model  of  order  35  and 
BJ  model  of  order  1 1  are  obtained.  For  ring2  sensor  (sensor 
on  opposite  side  of  ringl  sensor)  data  OE  model  of  order  45, 
ARX  model  of  order  12,  ARMAX  model  of  order  17,  BJ 
model  of  order  14,  N4SID  model  of  order  13  and  PEM  model 
of  order  22  are  obtained.  Table  I  gives  the  structure  of  all  the 
linear  models.  In  table  I  y  is  output,  t  is  time,  F(q), 

A(q),  C(q ),  D(q),  are  polynomials,  q  is  a  backward  shift 
operator,  u  is  input,  nk  is  delay  and  e  is  error  [12].  For  State- 
Space  models:-  x  is  state,  t  is  time,  Ts  is  sampling  time,  u  is 
input,  e  is  error,  A,  B,  K,  C,  and  D  are  system  matrices,  and  y 
is  output  [12]. 


predicted  output  as  a  weighted  sum  of  past  output  values  and 
current  and  past  input  values.  For  the  nonlinear  Hammerstein- 
Wiener  model  u(t)  and  y(t)  are  the  inputs  and  outputs  for 
the  system,  respectively.  /  and  h  are  nonlinear  functions  that 
corresponds  to  the  input  and  output  nonlinearity,  respectively. 
w(t)  and  x(t)  are  internal  variables.  w(£)  has  the  same 
dimension  as  u{t).  x(t)  has  the  same  dimension  as  y(t). 
B(q)  and  F(q)  in  the  linear  dynamic  block  are  polynomials 
in  the  backward  shift  operator.  For  nonlinear  Hammerstein- 
Weiner  model  with  deadzone  nonlinearity  estimator,  F  is  a 
nonlinear  function  of  x  with  the  properties  given  in  the  table 
II.  For  nonlinear  Hammerste in- Weiner  model  with  wavelet 
network  as  nonlinearity  estimator,  k(s)  as  a  wavelet 
function,  and  is  a  row  vector  such  that  /?fc(x  —  yk)  is  a 
scalar.  If  only  the  input  nonlinearity  is  present,  the  model  is 
called  the  Hammerstein  model.  If  only  the  output  nonlinearity 
is  present,  the  model  is  called  the  Wiener  model  [12]. 

TABLE  II 

NONLINEAR  MODELS  AND  THEIR  STRUCTURES 


Nonlinear  Model  Name  Nonlinear  Model  Structure 


TABLE  I 

LINEAR  MODELS  AND  THEIR  STRUCTURES 


Linear  Model  Name 

Linear  Model  Structure 

Output  Error 

y(t) 

B(q) 

ARX 

A(q)y(t) 

=  B(q)u(t  —  nk )  +  e(t) 

ARMAX 

A(q)y(t) 

=  B(q)u(t-nk ) 

+  c(q)e(t) 

Box-Jenkins 

y(0 

=  mu°  k) 

C(q) 

+  Wf(t) 

State-Space  -  subspace  method 

x(t  +  Ts) 

=  Ax(t )  +  Bu(t)  +  Ke(t) 

y(t) 

=  Cx(t )  +  Du(t)  +  e(t) 

State-Space  -  prediction  error/maximum 
likelihood  method 

x(t  +  Ts) 

=  Ax(t)  +  Bu(t)  +  Ke(t) 

y(t) 

=  Cx(t)  +  Du(t)  +  e(t) 

Table  II  gives  the  structures  of  nonlinear  models.  The 
nonlinear  models  for  the  motor  point  data  are  obtained  as,  the 
nonlinear  ARX  with  a  sigmoidnet  nonlinearity  estimator, 
nonlinear  Wiener-Hammerstein  models  with  a  piecewise 
linear  nonlinearity  estimator,  and  a  saturation  based  estimator. 
For  each  ringl  and  ring2  signal  data  sets,  five  nonlinear 
Wiener-Hammerstein  models  with  nonlinearity  estimators  as 
‘piecewise  linear  -  pwlinear,’  ‘sigmoidnet,’  ‘saturation,’ 
‘deadzone,’  and  wavelet  network’  are  obtained. 

In  table  II  /  is  a  nonlinear  function  for  the  nonlinear  ARX 
model.  Inputs  to  /  are  model  regressors.  yp(t)  is  the 


Nonlinear  ARX  -  yp(t)  =  f(y(t  —  l),y(t  — 
Sigmoidnet  2 ),y(t  -  3),  -  l),u(t  - 

2),...); 

/(z)  =  e_z+i  is  sigmoid  function. 


Nonlinear  HW  -  Piecewise 
Linear  -  pwlinear 


w(t)  =  f(u(t )), 


y(t)  =  ft(*(0); 


Nonlinear  HW  -  Sigmoidnet  yP(0  —  f(y(t  l),y(t 

2),y(t  —  3),  —  l),u{t  — 

2),...); 

/(z)  =  e_z+i  is  sigmoid  function. 


Nonlinear  HW  -  Saturation 


w(t)  =  f(u(t )), 

V (t)  =  w(t), 

y(0  =  Km); 


Nonlinear  HW  -  Deadzone 


Nonlinear  HW  -  Wavelet 
Network 


y  =  F(x)  ; 

a  <  x  <  b,  F{x )  =  0 
x  <  a,  F(x)  =  x  —  a 
x  >  b,  F(x )  =  x  -  b 


n 

gW  =  ’YjakK(Pk(x-Yk)) 

k= l 


IV.  Data  Fusion  Algorithm 

A  data  fusion  algorithm  with  an  adaptive  probability  of  the 
Kullback  information  criterion  (KIC)  is  used  to  fuse  the 
outputs  of  different  models  obtained  [13].  Kullback 
information  criterion  (KIC)  is  an  asymmetric  measure  for  the 
model  selection.  Kullback’ s  symmetric  or  J-divergence  is  the 
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sum  of  two  directed  divergences,  which  is  the  measure  of  the 
models  dissimilarity  and  given  by  Equation  (2),  [14]. 

KIC(pi)  =^\ogRi+^^-nip(^^+  g(n),  (2) 

where  g(ri)  =  n  *  log  (n/ 2). 

In  this  research  work,  we  are  obtaining  multi-  linear  and 
nonlinear  models  using  SI  for  the  sEMG  sensor  for  the  ring 
finger  motor  unit,  and  two  adjacently  placed  sensors,  ringl 
and  ring2.  The  probability  of  each  model  is  computed  using 
the  SI  model  fit  value,  which  is  given  by  [1  —  \Y  —  Y  \/\Y  — 
Y  |  ]  *  100.  The  fusion  of  outputs  and  adaptive  KIC 
probability  is  shown  in  Figure  3  which  is  followed  by  the  data 
fusion  algorithm. 


Figure  3.  Adaptive  KIC  Probability  Based  Data  Fusion. 


The  following  fusion  algorithm  as  given  by  [13]  is 
applied  for  data  fusion  of  the  outputs  of  different  linear  and 
nonlinear  models: 

1)  Identify  models  Mlf  M2,  —,Mk  using  sEMG  data  ( u )  as 
input  and  force  data  (Y)  as  output,  for  k  number  of 
sensors  collecting  data  simultaneously. 

2)  Compute  the  residual  square  norm 

Ri  =  || Y  -  Oi0i||2  =  || Y  -  Y\\,  where  0;  =  J-1cDi7Y, 

and 
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3)  Calculate  the  model  criteria  coefficient  using  Equation  (2). 

4)  Compute  the  model  probability  p(Mj|Z)  =  fce  l_t„  where 

£;= ie  1 

l  is  model  selection  criterion,  i.e.  KIC(pi ). 

5)  Compute  the  fused  model  output  Yf  =  £f=1p(Mj|Z)fj. 

6)  Compute  the  overall  model  from  Yf  and  force  data. 

Here  all  the  computation  from  step  2)  to  6)  is  adaptive  i.e. 
the  residual  square  norm,  A7C(pj),  model  probability 
p(Mj|Z),  and  fused  model  output  Yf  are  being  updated  with 
time  or  for  each  data  point. 

v.  Results  and  Discussion 

An  adaptive  KIC  probability  based  data  fusion  algorithm  is 
applied  to  linear  and  nonlinear  models  separately  for  the 
models  obtained  using  three  sets  of  input  and  output  data  for 
three  sEMG  sensors.  First  the  fusion  based  output  for  each 
sensor  is  obtained  and  then  the  three  fusion  based  outputs 
from  three  sEMG  sensors  are  fused  to  get  the  improved 
estimates  of  the  skeletal  muscle  force.  The  following  plots 
show  the  improvement  in  the  force  estimates  in  succession 
with  different  sensors  separately  and  then  combined.  All  the 
plots  have  the  measured  skeletal  muscle  force  signal  in  green 
and  the  estimated  force  signal  in  red  color.  Figure  4  is  a  plot 
of  measured  and  Chebyshev  type  II  filtered  finger  force  data 
with  the  adaptive  KIC  probability  fusion  based  force  for  the 
first  sensor  named  as  ringl.  The  two  signals  are  very  clear  and 
the  estimated  signal  shows  good  follow  up  of  the  measured 
signal.  Figure  5  shows  the  measured  and  Chebyshev  type  II 
filtered  skeletal  muscle  force  with  the  adaptive  KIC 
probability  fusion  based  force  signals  for  the  second  sensor 
named  as  ring2.  It  is  evident  that  the  measured  signal  has  a 
very  close  follow  up  by  the  estimated  signal. 


Chebyshev  Type  II  Filtered  Vs.  Linear-NL  Models  -  Adaptive  KIC  Probability  -  Ringl 


Figure  4.  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability 
Based  Force  from  Finear  and  Nonlinear  Models  for  Ringl 
Sensor. 

Figure  6  shows  the  measured  and  Chebyshev  type  II 
filtered  skeletal  muscle  force  with  the  adaptive  KIC 
probability  fusion  based  force  signals  for  third  sensor  placed 
on  the  motor  point  of  ring  finger.  The  results  for  this  sensor 
are  best  among  the  three  based  of  the  used  three  sensor  data 
and  it  is  evident  that  the  measured  signal  has  a  very  close 
follow  up  by  the  estimated  signal. 
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Chebyshev  Type  II  Filtered  Ys.  Linear-NL  Models  -  Adaptive  KIC  Probability  -  Ring2 


Figure  5.  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability 
Based  Force  from  Linear  and  Nonlinear  Models  for  Ring2 
Sensor. 


Chebyshev  Type  II  Filtered  Vs.  Motor  Point  -  Linear-NL  Models  -  Adaptive  KIC  Probability 


Figure  6.  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability 
Based  Force  from  Linear  and  Nonlinear  Models  for  Motor 
Point  Sensor. 


Chebyshev  Type  II  Filtered  Vs.  Ringl-Ring2-MotorPoint  -  Linear-NL  Models  -  Adaptive  KIC  Probability 


Figure  7.  Chebyshev  Type  II  vs.  Adaptive  KIC  Probability 
Based  Force  from  Linear  and  Nonlinear  Models  for  Ringl, 
Ring2  and  Motor  Point  Sensor  Combined. 

Finally  the  estimated  skeletal  muscle  force  from  three 
sEMG  sensors  is  further  fused  with  adaptive  KIC  probability 
based  data  fusion  algorithm.  Figure  7  shows  the  comparison 
of  the  measured  and  Chebyshev  type  II  filter  with  the  final 
adaptive  KIC  probability  based  fusion  skeletal  muscle  force 


estimate  using  three  sEMG  sensors.  The  final  result  for  three 
sensors  is  the  best  estimate  of  skeletal  muscle  force  and  it  is 
evident  that  the  measured  signal  has  a  very  close  follow  up  by 
the  estimated  signal. 

The  results  show  that  there  is  a  decrease  in  the  percentage 
error  from  the  linear  and  nonlinear  model  fusion  based 
separate  outputs  to  the  fusion  based  combined  output  of  linear 
and  nonlinear  models  combined  for  all  three  sensors. 
However,  these  results  show  that  there  is  a  1 6%  improvement 
in  the  mean  fit  value  of  the  motor  point  signal  models  with 
the  adaptive  KIC  probability  based  data  fusion  algorithm  for 
multi-sensors  (three  in  this  case).  Figure  8  and  9  show  the 
validation  plots  of  this  approach  for  two  separate  sets  of 
sEMG  and  skeletal  muscle  force  data  where  green  signal  is 
the  Chebyshev  type  II  filtered  measured  force  signal  and  the 
blue  signal  is  estimated  using  this  approach. 


Validation  Plot  -  Separate  Experiment  Data  1 


2  4  6  8  10  12 

Time  (60  Seconds)  x  104 


Figure  8.  Validation  Plot  1  -  Chebyshev  Type  II  vs.  Estimated 
Force  Signal  using  Adaptive  KIC  Probability. 


Validation  Plot  -  Separate  Experiment  Data  2 


Figure  9.  Validation  Plot  2  -  Chebyshev  Type  II  vs.  Estimated 
Force  Signal  using  Adaptive  KIC  Probability. 

vi.  Conclusion  and  Future  Work 

Different  linear  and  nonlinear  models  are  obtained  using 
system  identification  (SI)  for  three  sets  of  data.  Linear  and 
nonlinear  models’  outputs  are  fused  separately  for  each  sensor 
and  then  fused  together  to  get  the  final  force  estimate  for  each 
sensor.  The  net  force  estimates  for  each  sensor  is  then  applied 
with  adaptive  KIC  probability  based  data  fusion  algorithm. 
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The  final  estimates  of  the  skeletal  muscle  force  gives 
improved  results.  Future  work  will  focus  on  the  improved  data 
collection  techniques  and  capturing  data  at  higher  sampling 
rate  (6000  samples  per  second).  Also,  we  would  design  new 
model  selection  criteria  using  different  model  selection 
criteria  such  as  Akaike  Information  Criterion  (AIC),  Kullback 
Information  Criterion  (KIC)  and  the  Bayesian  Information 
Criterion  (BIC). 
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Abstract — Precise  and  effective  prosthetic  control  is  important  for 
its  applicability.  Two  desired  objectives  of  the  prosthetic  control 
are  finger  position  and  force  control.  Variation  in  skeletal  muscle 
force  results  in  corresponding  change  of  surface 
electromyographic  (sEMG)  signals.  sEMG  signals  generated  by 
skeletal  muscles  are  temporal  and  spatially  distributed  that  result 
in  cross  talk  between  adjacent  sEMG  signal  sensors.  To  address 
this  issue,  an  array  of  nine  sEMG  sensors  is  used  with  a  force 
sensing  resistor  to  capture  muscle  dynamics  in  terms  of  sEMG 
and  skeletal  muscle  force.  sEMG  and  skeletal  muscle  force  are 
filtered  with  a  nonlinear  Teager-Kaiser  Energy  (TKE)  operator 
based  nonlinear  spatial  filter  and  Chebyshev  type-II  filter 
respectively.  Multiple  Takagi-Sugeno-Kang  Adaptive  Neuro 
Fuzzy  Inference  Systems  (ANFIS)  are  obtained  using  sEMG  as 
input  and  skeletal  muscle  force  as  output.  Outputs  of  these 
ANFIS  systems  are  fitted  with  smoothing  spline  curve  fitting.  To 
achieve  better  estimate  of  the  skeletal  muscle  force,  an  adaptive 
probabilistic  Kullback  Information  Criterion  (KIC)  for  model 
selection  based  data  fusion  algorithm  is  applied  to  the  smoothing 
spline  curve  fitting  outputs.  Final  fusion  based  output  of  this 
approach  results  in  improved  skeletal  muscle  force  estimates. 

Keywords  -  sEMG;  TKE;  ANFIS;  KIC 

I.  Introduction 

In  the  United  States,  1.6  million  people  lived  with 
amputations  during  2005  [1].  This  increase  reflects  on  the  war 
effects  in  Afghanistan  and  Iraq  since  2003  [2].  During  the 
same  period  of  time,  the  fields  of  prosthetic  design  and 
development  research  for  prosthesis  has  received  increased 
attention.  Precise  and  timely  control  of  prosthesis  is  critical  for 
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its  applicability,  which  necessitates  the  replacement  of 
conventional  algorithms  by  intelligent  systems  and  decision 
methods.  Therefore,  a  natural  means  of  interface  between  the 
user  and  prosthesis  is  required.  sEMG  signals  from  skeletal 
muscles  are  a  convenient  means  for  users  who  do  not  want  to 
deal  with  additional  surgeries  and  regular  hygiene  problems. 
sEMG  signals  are  simply  electric  voltages  ranging  between  -5 
and  +5  [mV]  which  can  be  recorded  on  the  skin  surface  of  a 
hand  amputee’s  forearm.  Skeletal  muscles  generate  a  base 
sEMG  even  without  a  limb  movement  which  changes  with 
different  movements  and  applied  forces.  This  makes  sEMG 
signals  useful  for  position  and  force  control  of  prosthesis. 
Presently  available  prosthetic  hands  have  no  tactile  or 
proprioceptive  feedback  for  grasping  and  only  user’s  direct 
vision  as  a  sensory  feedback  is  available  for  its  position 
control.  Afore  mentioned  reasons  cause  30-50%  of  the  upper 
extremity  amputees  to  choose  not  to  use  their  prosthetic  hand 
on  a  regular  bases  [3-4]. 

Effective  grasping  of  any  object  requires  the  knowledge  of 
force  and  its  feedback.  Since  the  hand  amputee  uses  only 
sEMG  signals  collected  from  skeletal  muscles,  there  is  no 
hand  force  information  directly  available.  sEMG  is  a  measure 
of  the  nervous  excitation  that  ultimately  is  responsible  for 
activating  the  skeletal  muscles,  which  in  turn  generate  the 
finger  forces  with  the  help  of  the  link  kinematics  of  human 
hand.  Therefore,  a  model  or  estimation  scheme  is  needed  for 
the  dynamic  estimation  of  skeletal  muscle  force  from  sEMG 
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signals.  There  have  been  numerous  efforts  in  this  direction  and 
some  of  the  recent  are  evident  in  the  research  work  of  [5-10]. 

The  present  work  proposes  a  novel  scheme  of  skeletal 
muscle  force  identification  using  sEMG  signals.  An  array  of 
nine  sEMG  sensors  is  used  to  record  sEMG  signals  and 
skeletal  muscle  force  signals  recorded  with  a  force  sensing 
resistor  from  the  arm  of  a  healthy  subject,  which  respectively 
is  explained  in  detail  in  Section  II  and  III.  Eight  Takagi- 
Sugeno-Kang  ANFIS  with  different  membership  functions 
each  with  a  rule  base  of  ten  are  designed.  The  training  of  each 
ANFIS  is  done  using  a  hybrid  optimization  method  with  zero 
error  tolerance  and  40  epochs.  Each  of  the  force  signals 
simulated  using  these  ANFIS  and  measured  force  signals  are 
applied  to  a  smoothing  spline  curve  fitting  method.  Figure  1 
shows  the  flow  chart  of  the  proposed  approach.  Final 
estimated  force  signal  shows  good  match  with  the  measured 
signal.  This  approach  is  tested  on  two  different  data  sets  of  45 
and  60  seconds  duration  and  the  results  seem  promising.  The 
paper  is  organized  in  eight  sections.  Present  section  follows 
the  experimental  set-up  and  then  signal  processing,  ANFIS 
modeling,  smoothing  spline  curve  fitting,  data  fusion,  results 
and  discussion  and  conclusion  and  future  work  is  presented 
respectively. 


V  V  v  V 


V 


Figure  1 .  Flow  chart  of  the  approach  used  in  this  research  work. 


II.  Experimental  Set-Up 

Based  on  the  set-up  of  a  sEMG  sensor  array  in  [11],  an 
experiment  set-up  was  developed  using  DELSYS®  Bagnoli- 
16  EMG  system  with  nine  DE-3.1  sEMG  sensors  to  capture 
sEMG  signals  from  skeletal  muscles.  Here,  the  array  involves 
nine  sensors  covering  four  directional  spatial  distributions  of 
the  sEMG  signal.  The  middle  three  sEMG  sensors  were 
attached  directly  on  the  skin  surface  above  the  motor  point  of 
the  flexor  digitorum  superficialis  muscle  (FDS)  which  controls 
the  flexion  of  the  ring  finger.  Appropriate  sEMG  electrode 
attachment  point  for  the  flexion  of  the  ring  finger  was 
identified  using  a  wet  probe  muscle  stimulator  at  the  FDS 
(RICH-MAR,  HV  1000). 

Prior  to  placing  the  sEMG  sensors,  the  skin  surface  of  the 
subject  was  prepared  according  to  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK)  protocols.  The 
thumb  movement  was  restricted  in  this  experiment  using  a 
thumb  splint.  However,  the  corresponding  force  data  was 
captured  using  NI  ELVIS™  with  Interlink™  Electronics  FSR 
0.5”  circular  force  sensor.  Experimental  set-up  is  shown  in 
Figure  2,  where  9  sensors  are  shown  on  a  healthy  subject 
forearm,  holding  a  stress  ball  for  force  measurements.  The 
sEMG  and  finger  force  data  was  collected  at  a  sampling  rate 
of  2000  Hz  using  Lab  VIEW™  in  conjunction  with  DELSYS® 
Bagnoli-16  EMG  and  NI  ELVIS™.  With  this  experimental 
set-up  we  conducted  several  experiments  of  30  seconds,  45 
seconds  and  60  seconds  durations. 


Figure  2.  Experiment  set-up. 

III.  Signal  Processing 


The  recorded  sEMG  signal  is  filtered  using  nonlinear 
spatial  filtering  for  multichannel  surface  EMG.  As  given  in 
[11],  this  nonlinear  spatial  filter  is  derived  from  “Nonlinear 
Teager-Kaiser  Energy  (TKE)  Operator.”  As  given  in  [11],  the 
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TKE  operator  in  the  time  domain  on  sEMG  signal  is  defined 
by  Equation  (1). 

xV[x(n)]  =  x2(n )  -  x(n  +  1)  *  x(n  -  1).  (1) 

Here,  H7  is  the  TKE  operator  and  x(n)  is  the  time  domain 
sEMG  signal.  Based  on  the  TKE  operator,  the  four¬ 
dimensional  nonlinear  spatial  filter  is  given  by  Equation  (2). 

'Vd^xim,  n)]  =  4  *  x2(m,n )  -  x(m  —  l,n)  *  x(m  +  1  ,ri)  — 
x(m,  n  —  1)  *  x(m,  n  +  1)  -  x(m  —  1,  n  +  1)  *  x(m  +  1,  n  — 
1)  -  x(m  -  1,  n  -  1)  *  x(m  +  1,  n  +  1).  (2) 

Figure  3  shows  a  comparison  between  the  measured  and 
nonlinear  spatial  filtered  sEMG  signals. 

A  Chebyshev  type  II  infinite  impulse  response  (HR)  low 
pass  filter  with  a  550  Hz  pass  band  frequency  is  used  to  filter 
the  skeletal  muscle  force  signal.  Figure  4  shows  the  comparison 
between  the  measured  and  Chebyshev  type  II  filtered  skeletal 
muscle  force  signal. 


Figure  3.  sEMG  signal  from  ring  finger. 


Measured  Skeletal  Muscle  Force  Signal  -  Without  Filteration 
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Figure  4.  Skeletal  muscle  force  signal  from  ring  finger. 


In  1965  professor  Lofti  A.  Zadeh  introduced  the  concept  of 
“ Fuzzy  Sets ”,  which  are  sets  with  imprecise  amplitudes  [12]. 
Dr.  Zadeh  stated  that  the  “ membership ”  in  a  fuzzy  set  is  a 
matter  of  degree  and  not  a  matter  of  affirmation  or  denial.  For 
last  four  decades,  this  proposal  has  gained  recognition  as  an 
important  point  in  the  evolution  of  modern  concept  of 
imprecision  and  uncertainty  [12].  This  innovation  represents  a 
paradigm  shift  from  the  classical  sets  or  the  crisp  sets  to 
“ Fuzzy  Sets ”  [12]. 

Fuzzy  set  and  fuzzy  logic  theory  can  be  used  to  capture  the 
natural  phenomenon  of  the  imprecision  and  uncertainty  [12]. 
The  membership  function  is  the  characteristic  of  fuzzy  set, 
which  is  a  function  whose  range  is  an  ordered  membership  set 
within  the  closed  unit  interval  [12]. 

A  crisp  set  which  is  a  collection  of  distinct  objects  is 
defined  in  a  way  to  partition  objects  in  a  given  domain  of 
discourse  into  two  groups:  members  and  non-members  [12]. 
On  the  other  hand,  a  fuzzy  set  can  be  formed  by  assigning  a 
membership  value  (which  represents  the  degree  to  which  an 
object  belongs  to  a  fuzzy  set)  to  each  object  in  the  interval  of  0 
to  1  [12].  Fuzzy  membership  functions  can  be  of  different 
forms  such  as  triangular,  trapezoidal,  pi,  bell  and  Gaussian 
form. 

Fuzzy  Inferance  System  (FIS)  in  fuzzy  logic  can  be  used  to 
model  a  system  which  is  an  inference  system  based  on 
empirical  knowledge  based  linguistic  rules  [13].  The  fuzzy 
inference  system  is  composed  of  five  steps:  fuzzification  of 
the  input  variables,  application  of  the  fuzzy  operator  (AND  or 
OR)  in  the  precedent,  implication  from  the  precedent  to  the 
subsequent,  aggregation  of  the  subsequents  across  the  rules, 
and  defuzzification.  In  the  fuzzification  process,  the 
membership  values  coming  from  quantitative  measurements 
are  converted  to  linguistic  values  based  on  the  membership 
functions  [13]. 


0  12  3  4 


input  variable  "inputT1 

Figure  5.  Gaussian  membership  functions  for  sEMG. 
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Figure  5  shows  the  plot  of  Gaussian  membership  functions 
for  fuzzification  of  sEMG.  The  process  of  fuzzification 
followed  by  IF  ..  AND..  THEN  ..OR  ELSE  type  control  or 
relation/estimation  rules  [13].  The  output  of  the  rules  is 
computed  as  a  max-min  composition  [13].  In  the 
defuzzification  process,  the  output  of  the  rules  has  to  be 
converted  to  a  crisp  value  [13]. 


inputl  =  0 


output  =  4.38 


Figure  6.  Illustration  of  ANFIS  inference  mechanism. 
TABLE  I.  Membership  Functions 


S.  No. 

Membership  Function 

1. 

Gaussian  (GAUSSMF) 

2. 

Gaussian  combination  (GAUSS2MF) 

3. 

Difference  between  two  sigmoidal  membership  functions 
(DSIMF) 

4. 

Generalized  bell-shaped  (GBELLMF) 

5. 

II-shaped  (PIMF) 

6. 

Product  of  two  sigmoidally  shaped  membership  functions 
(PSIMF) 

7. 

Trapezoidal-shaped  (TRAPMF) 

8. 

Triangular-shaped  (TRIMF) 

Using  eight  different  membership  functions  types  we 
designed  eight  single-output  Sugeno-type  adaptive  neuro- 
fuzzy  inference  systems  (ANFIS)  using  a  grid  partition  on  the 
data.  The  output  membership  function  in  each  of  these  is 
linear.  Each  ANFIS  has  ten  rules  and  connection  type  is  ‘And’ 
with  a  weight  of  1.  For  the  ‘And’  method  ‘product’  and  for 
‘Or’  method  ‘probabilistic’  types  are  used  for  each  ANFIS. 
Deffuzification  is  achieved  using  weighted  average  (wtaver) 
from  each  rule.  Figure  6  shows  the  rules  for  ANFIS  with 
Gaussian  membership  functions  and  illustrate  the  ANFIS 
inference  mechanism.  The  membership  functions  used  are 
tabulated  in  Table  1. 


-( x-c )2 

A  Gaussian  function  is  given  by  /(x;  a,c)  =  e  2^2  for 
some  real  constants  c  and  o  >  0,  and  e  =  2.718.  Gaussian 
combination  membership  function  (GAUSS2MF)  depends  on 
the  c  and  a  parameters  [14].  The  sigmoidal  membership 
function  is  given  as  /(x;  a,  c )  =  i+g_a(x_c)  and  depends  on  the 

two  parameters  a  and  c  .  The  difference  between  two 
sigmoidal  membership  function  (DSIMF)  which  is  the 
difference  between  two  sigmoidal  functions  depends  on  the 
four  parameters  a1,c1,a2,  and  c2  [14].  Generalized  bell¬ 
shaped  membership  function  (GBELLMF)  is  given  as 

/(x;  a,  b,  c)  = - and  depends  on  three  parameters  a,  b, 

l+l— I 

I  a  I 

and  c  where  parameter  a  and  b  decide  the  width  of  the  curve 
and  parameter  c  locates  the  center,  usually  b  >  0  [14].  Pi- 
shaped  membership  function  (PIMF)  is  given  by 


/(x;  a,  b ,  c,  d ) 


2+b 


\ 


and 


evaluated  at  the  points  given  by  vector  x.  The  “feet”  of  the 
curve  is  determined  by  parameters  a  and  d  whereas 
parameters  b  and  c  decides  its  “shoulders”  [14].  The  Product 
of  two  sigmoidally  shaped  membership  functions  (PSIMF)  is 
simply  the  products  of  two  sigmoidal  functions  as  given 
before.  The  trapezoidal  shaped  membership  function 
(TRAPMF)  is  given  by 


/(x;  a,  b,  c,  d ) 


'  0, 
x-a 

b-a  1 

1, 

d-x 
d-c 
v  0, 


x  <  a 
a  <  x  <  b 


\ 


b  <  x  <  c 
c  <  x  <  d 
d  <  x  / 


which  depends  on 


four  scalar  parameters  a,  b,c,  and  d .  The  triangular  shaped 
membership  function  (TRIMF)  is  given  by 
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/  O;  a,  b,  c)  = 


°' 

x-a 

b-a  ’ 


x  <  a  \ 
a  <  x  <  b 


c-x 

— b  <  x  <  c 

c-b 

C  <  X 


which  depends  on  the 


L—U 

\  0, 

scalar  parameters  a,  b  and  c  [14]. 


/ 


small.  In  this  work  the  smoothing  parameter  p  is  chosen  as 
0.92  [15].  Figure  7  shows  the  analysis  results  for  smoothing 
spline  curve  fitting  to  the  output  of  ANFIS  with  Gaussian 
membership  function. 


VI.  Data  Fusion 


V.  Smoothing  Spline  Curve  Fitting 

MATLAB®  Curve  Fitting  Toolbox™  is  a  useful  tool  for 
fitting  curves  and  surfaces  to  acquired  data  [15].  It  can  be  used 
to  do  exploratory  data  analysis,  preprocessing  and  post¬ 
processing  of  the  data,  comparing  candidate  models,  and  for 
removing  outliers  [15].  Weights  for  each  smoothing  spline 
curve  fit  are  calculated  by  subtracting  the  ANFIS  output  from 
the  measured  skeletal  muscle  force  output.  A  graphical  user 
interface  is  used  to  fit  the  smoothing  spline  curve  to  each 
output  from  eight  ANFIS  with  respective  weights.  Smoothing 
spline  which  is  a  piecewise  polynomial  computed  from  a 
smoothing  parameter  (p)  of  0.92  is  fitted  to  each  of  eight 
outputs  of  ANFIS.  Smoothing  parameter  (p)  is  a  number 
between  0  and  1 .  Change  in  the  value  of  p  from  0  to  1  results 
in  the  change  in  smoothing  spline.  For  p  =  0  the  smoothing 
spline  is  a  least- square  straight-line  approximation  to  the  data 
whereas  for  p  =  1  it  gives  the  '’natural”  cubic  spline 
interpolant  to  the  data  [15]. 


Analysis  of  fit  "fit  1 "  for  dataset  "a_e_40GAUSS2MF  with  wtl_2GA40" 


Figure  7.  Analysis  results  for  smoothing  spline  curve  fitting  to  the  output  of 
ANFIS  with  Gaussian  membership  function. 

For  the  specified  weight  (w;)  and  smoothing  parameter  (p) 
a  smoothing  spline  s  is  formed  which  minimizes  the 
parameter  J  given  by  Equation  (3). 

/  =  pliWiCyt  ~s(Xi))2  +  (1  ~P)  I (.j^)2dx,  (3) 

where  xt  and  yt  are  predictor  and  response  data  respectively. 
The  smoothing  parameter  p  should  be  chosen  to  make  the 

error  E(s)  =  Wi(yt  —  s(xt))2  and  roughness  J  (^r)2^x 


Data  fusion  of  multiple  outputs  of  smoothing  spline  curve 
fittings  is  done  by  assigning  a  particular  probability  to  each 
individual  output  [16].  This  method  gives  good  estimation  of 
the  skeletal  muscle  force.  The  goodness  of  the  fit  of  each 
smoothing  spline  output  can  be  measured  using  System 
Identification  (SI)  model  fit  value  which  is  given  by  Equation 

(4). 

FIT  =  [1-|  Y-Y  |/|  Y  -  7  |  ]  *  100  (4) 

In  Equation  (4)  Y  is  measured,  Y  is  estimated  and  7  is 
mean  output  signal.  The  model  selection  criterion  used  in  this 
paper  is  Kullback  Information  Criterion  (KIC)  which  has 
shown  to  perform  well  for  sEMG  sensor  data  fusion  [7-10]. 
The  sum  of  two  directed  divergences,  which  is  the  measure  of 
the  models  dissimilarity,  is  known  as  Kullback’ s  symmetric  or 
J-divergence  [17],  as  given  by  Equation  (5). 

KIC(Pt)  =^logRl+^^-nip^YJb+  g(n),  (5) 

where  g(n )  =  n  *  log(n/ 2). 

The  fusion  algorithm  as  given  by  [16]  is  applied  for  data 
fusion  of  the  outputs  of  eight  smoothing  spline  curve  fit 
outputs  which  are  obtained  from  eight  different  ANFIS 
models: 

(I)  Find  smoothing  spline  curve  fit  outputs  Mlf  M2, ... ,  Mk  for 
k  number  of  outputs. 

(II)  Compute  the  residual  square  norm 

Ri  =  \\Y-0i  ©ill2  =  \\Y~?  II2, 

where  and 
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(III)  Calculate  the  model  criteria  coefficient  using  Equation 
(5). 

e-h 

(IV)  Compute  the  model  probability  p(Mj|Z)  =  — — where 

Hj=ie  J 

l  is  model  selection  criterion,  i.e.  K/C(pj). 

(V)  Compute  the  fused  model  output  Yf  =  Yif=i  p(Mi|Z)Yi. 

(VI)  Compute  the  overall  model  from  Yf  and  skeletal  muscle 
force  data. 

Here  all  the  computation  from  step  (II)  to  (VI)  is  adaptive 
i.e.  the  residual  square  norm,  KlC(Pi)  ,  model  probability 


Page  249  of  531 


pCMjZ),  and  fused  model  output  Yf  are  being  updated  with 
time  or  for  each  data  point.  Figure  8  shows  the  flow  chart  for 
fusion  of  outputs  and  adaptive  probability  of  KIC. 


Figure  8.  Flow  chart  for  fusion  of  outputs  and  adaptive  probability. 

VII.  Results  and  Discussion 

The  proposed  method  gives  good  estimation  of  the  skeletal 
muscle  force.  A  data  set  of  45  seconds  is  used  to  estimate  the 
output.  Figure  9  shows  the  measured  and  estimated  skeletal 
muscle  force  using  this  method. 


ANFIS  -  CURVE  FITTING  -  DATA  FUSION  OUTPUT 


Figure  9.  Plot  of  measured  and  proposed  method  estimated  skeletal  muscle 

force. 


Figures  10  and  11  are  the  validation  plots  for  45  seconds 
and  60  seconds  of  data  sets  respectively.  All  of  these  three 
plots  show  a  very  close  follow  up  of  the  measured  signal  by 
the  estimated  signal  using  this  approach.  Figure  12  shows  the 
percentage  error  between  the  measured  force  signal  and 
estimated  finger  force  signal  for  45  seconds  of  data  set.  The 
percentage  error  reduced  to  the  lower  values  and  the 
maximum  percentage  error  value  is  1.7  percent.  The  mean  fit 
value  between  the  measured  and  estimated  output  is  91.3 
percent. 


Validation  Plot  1  -  ANFIS -CURVE  FITTING -DATA  FUSION 


Figure  10.  Validation  plot  1  using  different  data  set. 
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Figure  1 1 .  Validation  plot  2  using  different  data  set. 
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Figure  13  shows  the  percentage  error  plot  for  different  data 
set  of  60  seconds  duration.  Percentage  error  reduced  to  lower 
values  and  the  maximum  percentage  error  value  is  3.7  percent. 
The  mean  fit  value  between  the  measured  and  estimated 
output  is  40  percent.  The  mean  fit  value  for  another  45 
seconds  of  data  set  is  73  percent. 


Percentage  Error  -  Validation  2  Data 


Time 

Figure  13.  Percentage  error  plot  for  60  seconds  data  set. 


VIII.  Conclusion  and  Future  Work 

Using  sEMG  and  skeletal  muscle  force  as  input  and  output 
data,  eight  ANFIS  were  designed  and  tested.  Output  of  these 
ANFIS  were  fitted  with  smoothing  spline  curve  fit  and 
resultant  outputs  were  fused  using  a  KIC  based  data  fusion 
algorithm.  This  approach  has  been  validated  using  two 
separate  data  sets.  Results  are  promising  and  show  good 
estimation  of  skeletal  muscle  force  which  can  be  applied  to 
design,  application  and  improvement  of  prosthetic  hands  for 
amputees. 

Future  work  will  address  the  improvement  of  data 
collection  techniques,  sEMG  sensor  design,  use  of  this 
approach  for  finger  angles  and  stiffness  estimation  from 
sEMG  signals.  Different  modeling  techniques  with  this 
approach  and  combined  probability  of  different  model 
selection  criteria  such  as  Akaike  Information  Criterion  (AIC), 
Kullback  Information  Criterion  (KIC)  and  the  Bayesian 
Information  Criterion  (BIC)  together  will  give  improved 
results. 
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TOWARDS  SMART  PROSTHETIC  HAND:  ADAPTIVE  PROBABILITY 
BASED  SKELETAN  MUSCLE  FATIGUE  MODEL 
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Abstract  -  Skeletal  muscle  force  can  be  estimated  using  surface 
electromyographic  (sEMG)  signals.  Usually,  the  surface  location 
for  the  sensors  is  near  the  respective  muscle  motor  unit  points. 
Skeletal  muscles  generate  a  spatial  EMG  signal,  which  causes 
cross  talk  between  different  sEMG  signal  sensors.  In  this  study,  an 
array  of  three  sEMG  sensors  is  used  to  capture  the  information  of 
muscle  dynamics  in  terms  of  sEMG  signals.  The  recorded  sEMG 
signals  are  filtered  utilizing  optimized  nonlinear  Half-Gaussian 
Bayesian  filters  parameters,  and  the  muscle  force  signal  using  a 
Chebyshev  type-II  filter.  The  filter  optimization  is  accomplished 
using  Genetic  Algorithms.  Three  discrete  time  state-space  models 
are  obtained  using  system  identification  and  modal  transformation 
for  three  sets  of  sensors  for  single  motor  unit.  The  outputs  of  these 
three  models  are  fused  with  a  probabilistic  Kullback  Information 
Criterion  (KIC)  for  model  selection.  The  final  fused  output  is 
estimated  with  an  adaptive  probability  of  KIC,  which  provides 
improved  force  estimates. 


I.  Introduction 

National  Limb  Loss  Information  Center  [1]  reported  in  2002 
that  1.2  million  people  live  with  amputations.  Since  then,  this 
number  is  increasing  due  to  ongoing  wars  in  Iraq  and 
Afghanistan.  However,  prostheses  can  significantly  improve  the 
lives  for  these  people. 

Recent  research  efforts  have  been  active  towards  creating 
surface  electromyographic  (sEMG)  based  prosthetics  for  hand 
amputees.  The  recorded  sEMG  signal  is  used  as  an  input  to 
activate  the  prosthesis.  The  central  nervous  system  activates 
and  controls  the  EMG  signals  which  depend  on  the  flow  of 
specific  ions  including  sodium  ( Na+ ),  potassium  ( K+ )  and 
calcium  (Ca++)  resulting  in  the  action  potentials  in  nerves  and 
their  respective  skeletal  muscle  fibres.  A  potential  difference 
develops  across  neuronal  membranes  because  of  this  ion 
exchange  and  this  can  be  measured  as  an  electrical  voltage 
change  [2].  sEMG  signals  are  collected  from  the  skin  as  an 
electric  voltage  ranging  between  -5  and  +5  mV. 
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The  control  of  the  myoelectric-based  prosthesis  is  difficult 
because  of  the  dynamic  nature  of  the  EMG  signal  [1]  due  to 
varying  motor  unit  recruitment,  crosstalk,  and  biochemical 
interaction  within  the  muscular  fibres.  In  addition,  sEMG 
changes  continuously  due  to  the  onset  and  progression  of 
muscle  fatigue  [1]. 

Muscle  fatigue  is  complex  in  nature  and  results  in  failure  to 
maintain  the  required  force  level  [3].  Reasons  for  fatigue  can  be 
the  peripheral  changes  at  the  muscle  level  or  an  inadequate 
output  from  the  central  nervous  system  to  stimulate 
motoneurons  [3].  The  amount  of  force  generated,  duration  of 
each  contraction,  and  the  rest  period  between  two  contractions 
has  a  direct  influence  on  the  muscle  fatigue  rate  [4].  Muscle 
fibre -type  distribution  [5],  nerve  conduction  velocity  of 
fatiguing  muscles,  or  even  factors  within  the  central  nervous 
system  (CNS)  affect  EMG  signals  [6].  EMG  analysis  is  a  well- 
accepted  method  for  muscle  fatigue  assessment  [6-10]. 

In  1977,  L.  Lindstrom,  R.  Kadefors  and  I.  Petersen 
developed  a  method  that  measures  the  localized  muscle  fatigue 
based  on  the  power  spectrum  analysis  using  myoelectric  signals 
[11].  This  approach  permits  real-time  investigations  and  can 
yield  statistically  based  criteria  for  the  occurrence  of  fatigue. 
Rate  of  fatigue  development  and  changes  in  muscle  action 
potential  conduction  velocity  were  used  to  interpret  the  findings 
[11-13].  Additional  recruitment  of  motor  units,  synchronization 
of  active  motor  units  along  the  muscle  fibres,  and  a  decrease  in 
conduction  velocity  are  reflected  in  the  EMG  signal  as  an 
increase  of  amplitude  in  time  domain  and  a  decrease  of  medium 
frequency  in  frequency  domain  [14-15]. 

The  joint  analysis  method  using  EMG  amplitude  and 
spectrum  (JASA)  allows  distinguishing  between  the  difference 
of  fatigue-induced  and  force  related  EMG  changes. 
Simultaneous  changes  in  the  EMG  amplitude  and  spectrum  are 
considered  in  the  JASA  approach  [16].  The  JASA  principle 
states  that  muscle  force  decreases  because  of  the  decrease  in 
EMG  amplitude  [16].  Fatigue  can  occur  because  of  continuous 
high  frequency  stimulation  or  because  of  titanic  stimulation. 
The  Flow  Chart  of  the  work  in  this  paper  is  shown  in  figure  1 . 

This  paper  addresses  the  issue  of  skeletal  muscle  fatigue 
that  is  a  dynamic  phenomenon.  The  modeling  is  based  on 
system  identification,  where  mathematical  relations  are  inferred 
from  experimental  data.  The  data  consists  of  sEMG  signals  and 
hand/fmger  force  generated  by  healthy  subjects.  The  recorded 
three-sEMG  signals  are  filtered  using  nonlinear  Half-Gaussian 
Bayesian  filters  with  optimized  filter  parameters,  whereas  the 
skeletal  muscle  force  signal  is  pre-processed  using  a  Chebyshev 
type-II  filter  [1,17]. 
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Figure  1 :  The  Flow  Chart  of  the  Work  in  this  Paper. 


A  simple  Genetic  Algorithm  code  is  used  to  optimize  the 
Bayesian  filter  parameters.  Using  an  input/output  approach,  the 
EMG  signal  is  input  to  the  skeletal  muscle,  and  the  produced 
hand/fmger  force  constitutes  the  output.  Three  discrete  time 
state- space  models  are  obtained  using  system  identification  and 
modal  transformation  for  three  sets  of  sensors.  One  sensor  is 
placed  at  the  motor  unit  of  the  ring  finger  of  the  dominant  hand 
and  two  sensors  are  located  in  its  vicinity  as  shown  in  Figure  2. 
The  extracted  models  from  the  data  sets  are  fused  with  a 
probabilistic  Kullback  Information  Criterion  (KIC)  for  model 
selection.  The  final  fused  output  is  estimated  with  an  adaptive 
probability  of  KIC  that  gives  an  improved  skeletal  muscle  force 
estimates. 


II.  Experimental  set-up  and  pre-processing 

Both  sEMG  and  muscle  force  signals  were  acquired 
simultaneously  using  Lab  VIEW™  8.2  at  a  sampling  rate  of 
2000  Hz.  The  sEMG  data  capturing  was  aided  by  a  DELSYS® 
Bagnoli-16  EMG  system  with  DE-2.1  differential  EMG 
sensors.  The  corresponding  force  data  was  captured  using  NI 
ELVIS  with  Interlink  Electronics  FSR  0.5”  circular  force 


sensor.  The  experimental  set-up  is  shown  in  Figure  2.  One 
sEMG  sensor  was  placed  on  the  motor  point  of  ring  finger  and 
two  adjacent  to  the  motor  point  of  a  healthy  subject.  Prior  to 
placing  the  sEMG  sensors,  the  skin  surface  of  the  subject  was 
prepared  according  to  International  Society  of 
Electrophysiology  and  Kinesiology  (ISEK)  protocols. 


v  XI  ELVIS 


Figure  2:  Experimental  Set-Up. 

EMG  signal  is  modeled  for  the  conditional  probability  of 
the  rectified  EMG  signal  as  a  filtered  random  process  with 
random  rate.  The  likelihood  function  for  the  rate  evolves  in 
time  according  to  a  Fokker-Planck  partial  differential  Equation 

[17] .  Here,  a  and  /?  are  two  free  parameters,  a  is  the  expected 
rate  of  gradual  drift  in  the  signal,  and  /?  is  the  expected  rate  of 
sudden  shifts  in  the  signal.  These  two  free  parameters  of  the 
non-linear  “Half-Gaussian  filter  model”  are  optimized  for  the 
acquired  EMG  data  using  an  elitism  based  Genetic  Algorithm. 
A  Chebyshev  type  II  low  pass  filter  with  a  550  Hz  pass 
frequency  is  used  to  filter  the  force  signal. 

III.  System  identification  and  modal 

TRANSFORMATION 

An  alternative  to  physically  based  mathematical  modeling  is 
data  based  -‘system  identification  (SI)’-  that  can  be  applied  to 
different  systems  where  it  needs  sufficient  experimental  data 

[18] .  The  parametric  system  identification  process  yields  a 
dynamical  model  such  as  the  Auto  Regressive  with  exogenous 
input  (ARX)  or  Output  Error  (OE)  model,  this  model  gives  us 
the  system  in  mathematical  form.  In  this  work,  the  SI  approach 
is  used  with  the  myolelectric  signal  as  the  input  to  the  system 
and  as  output  the  intended  muscle  force. 

In  this  work,  we  are  using  an  output  error  (OE)  model 
structure  for  the  modeling  of  EMG  and  force  signals,  which  has 
been  documented  in  the  literature  to  perform  well  for  EMG 
signals  among  linear  parametric  models  [1].  The  OE  model  is 
by  Equation  (1). 

y(0  =  ~  nk )  +  e(t).  (1) 

A  common  OE  model  is  obtained  for  each  sensor  data  set  using 
one  set  of  model  parameters  (polynomial  order).  The  OE 
models  then  are  converted  into  a  state-space  formulation.  By 
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doing  this,  the  identification  properties  of  the  OE  model 
formulation  are  preserved  and  represented  in  the  state-space 
form.  A  modal  form  is  computed  by  using  following 
transformation: 

A  =  T-fn  =  t-1b,  r  =  ct.  (2) 

where  T  is  the  matrix  composed  of  the  eigenvectors  of  A,  and  D 
is  came  out  to  be  a  zero  matrix  for  the  cases  treated  in  this 
work.  Equation  (2)  refers  to  the  continuous  representation  of 
the  identified  models.  Using  the  modal  form  representation  for 
each  corresponding  data  set,  the  influence  of  fatigue  can  be 
observed  by  charting  the  coefficients  in  time  for  each  matrix. 
For  example,  the  coefficients  contained  in  the  transformed 
matrix  A  can  be  plotted  in  the  complex  s-plane.  This  tracking 
can  then  be  used  as  the  basis  for  constructing  polynomials  that 
connect  each  element  of  each  matrix  in  time  with  the 
corresponding  time  stamp.  Hence,  a  fatigue  model  in  state- 
space  description  is  constructed,  where  the  input  is  the  recorded 
EMG  signal  and  the  output  the  expected  force  generated  by  the 
hand  motion. 


5)  Compute  the  fused  model  output  Yf  =  £f=1p(Mj|Z)y;. 

6)  Compute  the  overall  OE  model  from  ?f  and  force  data. 

Here  all  the  computation  from  step  2)  to  step  6)  is  adaptive  i.e. 
the  residual  square  norm,  K/Cfe),  model  probability  p(MJZ),  and 
fused  model  output  Yf  are  being  updated  with  time  or  for  each 
data  point. 

V.  Results,  conclusion  and  future  work 

Based  on  previous  research  results  and  after  trying  different 
model  orders  for  sEMG/Force  data  an  optimal  model  order  of 
nb  =  2,  rif  =  2,  nk  =  1  is  used  in  this  work.  Equation  (4),  (5), 
and  (6)  give  three  resulting  discrete  time  state-space  sEMG- 
Force-Fatigue  models  for  three  sensors. 

xm  (k  +  1)  =  Am  (t)xm  (k)  +  Bm  ( t)EMGm  (/c); 

YHATm  (/c)  =  Cm  (t)xm  (k)  +  Dm  ( t)EMGm  (k) .  (4) 

Xl(k  +  1)  =  A^Oc)  +  B1EMG1  (/c); 

YHAT^k)  =  GxA/c)  +  D1EMG1  (k).  (5) 


IV.  Fusion  of  outputs  and  adaptive  probability  of 

kic 

Multiple  model  fusion  is  done  by  assigning  a  particular 
probability  to  each  individual  model  [19].  These  models  are 
obtained  using  System  Identification  (SI)  for  three  sEMG 
sensors  for  single  motor  unit.  SI  model  fit  value  gives  the 
probability  for  each  model,  which  is  given  by  [l-j^]*100. 

The  model  selection  criterion  used  in  this  paper  is  Kullback 
information  criterion  (KIC),  which  is  an  asymmetric  measure. 
The  sum  of  two  directed  divergences,  which  is  the  measure  of 
the  models  dissimilarity,  is  known  as  Kullback’s  symmetric  or 
J-divergence  [20],  as  given  by  Equation  (3). 

KIC(pi)  =  ^\ogR,  +  —  nip  (~~)  +  g(ri),  (3) 

where  g(n)  =  n  *  log^h/2). 

Following  fusion  technique  is  applied  for  sEMG  -  force 
identification  models,  which  is  based  on  [19]. 


x2(k  +  1)  =  A2(t)x2(k)  +  B2(t)EMG2(k ); 

YHAT2  (k)  =  C2  (fix  2  (k)  +  D2  ( t)EMG2  (k) .  (6) 


In  these  case  matrices  Dm>D1,  and  D2  are  zero  matrices,  whereas 
matrices  Am,Bm,Cm,A1,B1,C1,A2,B2,  and  C2  are  given  as: 
-1- c-  -  ['-■<»  '«")]: 

*-[?  £]=  *■-&]' 

B2  =  f^21  Si ;  and  C2  =  [c21(t)  c22(t)];  where  the  elements  of 
L^22  fUJ 


matrices  Am,Bm,Cm,A2,B2>  and  C2  are  function  of  time  and  the 
elements  of  matrices  AlfBu  and  C1  are  constants  for  this 
particular  data  set.  The  time  variables 

CLmi  (t),  am2  (t),  bml  (t),  bm2  (t),  cml  (t),  cm2  (t),  a21  (t),  a22  (t),  b21  (t),  b22  (t), 


c2 1  (t),  and  c22  (t)  are  given  by  quadratic  polynomials  obtained  by 
curve  fitting  the  tracked  modal  transformed  coefficients.  These 
time  variables  are  given  as: 


CLmi (0  =  0.00026t2  -  0.033t  -  11;  am2(t)  =  -0.0018 12  +  0.24t  -  33; 


1)  Identify  OE  models  M1;  M2,...,Mk  using  sEMG  data  ( u )  as 
input  and  force  data  (F)  as  output,  for  k  number  of  sensors 
collecting  data  simultaneously. 

2)  Compute  the  residual  square  norm 

where  and 
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3)  Calculate  the  model  criteria  coefficient  using  Equation  (3). 


4)  Compute  the  model  probability 


p(Mi\Z)  = 


Vi  '■ 


,  where  l  is  model  selection  criterion,  i.e. 


KIC(Pi). 


=  —0.002 12  +  0.33t  +  42;  fcm2(t)  =  -0.0017c2  +  0.33C  +  25; 
Cmi(t)  =  (3.2  *  10-7)c2  -  (4.2  *  10-5)t  +  0.71; 
cm2(t)  =  (4.6  *  10-8)t2  -  (5.8  *  10-6)t  -  0.71; 
a21  (c)  =  -(1.2  *  10~5)t2  +  0.0013C  -  0.47; 

a22(c)  =  0.001C2  -  0.037c  -  35;  f>21(C)  =  (6.4  *  10"6)C2  -  0.001C  +  0.3; 
b22  (C)  =  0.0023C2  -  0.12C  -  75; 

C21  (c)  =  -(1.8  *  lo-7)c2  +  (6.4  *  10“6)C  +  0.71;  and 
c22  (c)  =  -(2.1  *  10“9)C2  +  (2.4  *  10“7)C  -  0.71. 

Elements  of  matrices  and  C,  are  constants  for  this 

particular  data  set,  they  are  given  as: 

an  =  -0.0159;  a12  =  -43.1883;  bu  =  0.0090;  b12  =  -72.1875; 

cn  =  0.7147;  and  c12  =  -0.7071. 
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The  discrete  models  have  a  sampling  time  of  At  =  0.0005 
seconds.  The  duration  of  the  final  estimated  and  fused  output 
for  three  sensors  is  37.36  seconds.  Figure  3  depicts  the 
probability  and  10th  degree  curve  fitting  for  motor  point,  ring2 
and  ringl  sensor  signals. 


Figure  3:  10th  Degree  Curve  Fitting  -  Probability  Motor,  Ring2 
and  Ringl  Sensor  Signals. 

Figure  4  shows  the  original  and  estimated  fused  output, 
depending  on  the  error  between  the  two  later  follows  the 
original  signal  very  closely.  This  approach  identifies  a  fused 
fatigue  model  and  estimate  an  adaptive  probability  based  output 
that  follows  the  original  output  very  closely.  Outcome  of  this 
research  can  be  used  to  estimate  the  skeletal  muscle  force  of 
human  hand  for  prosthetic  hand  design,  application  and 
improvement. 


Figure  4:  Original  Output  Vs.  Estimated  Fused  Output  with 
Adaptive  Probability. 

In  future  by  improving  the  data  collection  techniques  and 
sEMG  sensor  design,  the  estimated  force  can  be  improved 
further. 
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Abstract:  -  This  paper  presents  a  novel  approach  to  control  the  motion  of  a  smart  prosthesis  using  surface 
electromyographic  (sEMG)  signals.  Currently,  all  sEMG  based  prosthetic  hands  are  controlled  based  on  pre¬ 
programmed  motion  sets,  which  are  initiated  when  some  threshold  value  of  the  measured  sEMG  signal  is 
reached.  In  this  paper,  we  present  an  approach  that  utilizes  System  Identification  (SI)  in  order  to  obtain  a 
dynamic  finger  angle  model.  Such  a  model  allows  for  instantaneous  control  for  the  finger  motions.  The 
algorithm  presented  relays  on  an  array  of  nine  sEMG  sensors.  The  sEMG  and  angle  data  is  filtered  using  a 
nonlinear  Teager-Kaiser  Energy  (TKE)  operator  based  nonlinear  spatial  filter  and  a  Chebyshev  type-II  filter 
respectively.  The  filtered  signals  are  smoothed  using  a  smoothing  spline  curve  fitting.  The  smoothed  sEMG 
data  is  used  as  input  and  the  respective  smoothed  finger  angle  data  is  used  as  output  for  a  system  identification 
routine  to  obtain  multiple  linear  and  nonlinear  models.  To  achieve  better  estimates  of  the  finger  angles,  an 
adaptive  probabilistic  Kullback  Information  Criterion  (KIC)  for  model  selection  based  data  fusion  algorithm  is 
applied  to  the  linear  and  nonlinear  model’s  outputs.  Final  fusion  based  output  of  this  approach  results  in 
improved  estimates  of  finger  angles. 

Key-Words:  -  sEMG,  Prosthetics,  System  Identification,  Teager-Kaiser  Energy  operator,  KIC. 


1  Introduction 

The  United  States  had  1.6  million  people  with 
amputations  during  2005,  [1].  This  number  is 
continuously  on  increase  due  to  the  ongoing  wars  in 
Afghanistan  and  Iraq,  [2].  Hence,  one  of  the  prime 
reasons  for  the  enlarged  attention  in  the  field  of 
prosthetic  is  to  facilitate  the  increased  demand  for 
efficiently  and  highly  dexterous  prosthesis.  Because 
of  the  lack  of  tactile  or  proprioceptive  feedback  for 
grasping  in  prosthesis,  almost  30-50%  of  the  upper 
extremity  amputees  choose  not  to  use  their 
prosthetic  hand  on  regular  bases  [3,  4].  The  control 
of  a  prosthetic  plays  a  key  role  in  the  usefulness  and 
ultimately  in  the  acceptance  of  the  device  by  a  user. 
Since  full  dexterity  is  still  out  of  reach  for  current 
prosthetics,  the  precision  and  effectiveness  of  a 
prosthetics  takes  on  greater  value.  The  controller 
implemented  must  be  capable  of  regulating  the 
intended  finger  forces  and  finger  motion  of  the 
artificial  hand.  The  reference  value  for  these  outputs 
is  given  by  the  amputee’s  mind/intention,  expressed 
by  the  generated  surface  electromyographic  (sEMG) 
signal  of  the  remaining  skeletal  arm  muscles.  In  this 
work,  we  assume  that  the  amputation  is  transradial, 
and  hence  sufficient  muscle  mass  is  accessible  for 
sEMG  data  acquisition.  EMG  can  be  measured  on 


the  surface  or  by  needle  electrodes  that  are  injected 
into  the  subjects  arm.  Choosing  the  sEMG  as  a 
control  input  eliminates  the  problems  associated 
with  surgeries  and  regular  hygiene  for  the  user  of 
implanted  electrodes.  sEMG  signal  amplitudes 
range  between  -5  and  +5  [mV].  The  sEMG  signal 
changes  with  different  limb  movements  and  applied 
forces.  This  justifies  the  use  of  sEMG  signals  for  the 
position  and  force  control  of  prosthesis.  Current 
prosthetics  using  EMG  or  sEMG  sensors  utilize  this 
data  by  computing  some  threshold  value  -  for 
example  the  RMS  value  -  to  activate  a  pre¬ 
programmed  motion  and/or  force  set  of  the  artificial 
hand.  The  resulting  motion  is  only  initiated  by  the 
user,  but  not  controlled  thereafter.  This  is  quite 
different  than  what  a  healthy  subject  uses  to  control 
his/her  hand.  For  non-amputees,  the  motion  of  the 
fingers  is  controlled  at  every  instance,  which  allows 
for  complex  motion  sets  to  be  executed.  To  mimic 
this  characteristic,  we  propose  to  use  dynamic 
models  relating  sEMG  data  with  finger  motion.  The 
potential  of  such  models  is  obvious  from  the 
operational  point  of  view,  but  also  allows  the 
incorporation  of  muscle  fatigue  dynamics  to  be 
included  in  the  control  algorithm  [9-13]. 
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Previous  research  works  give  different  methods 
to  extract  the  information  from  the  sEMG  signals 
[5].  These  methods  use  wavelet  analysis,  artificial 
neural  networks,  and  other  feature  extraction 
methods  to  make  use  of  sEMG  for  prosthetic  control 
[5].  Research  work  as  reported  in  [6]  presents 
sEMG  as  an  autoregressive  (AR)  model  with  the 
delayed  intramuscular  EMG  signal  as  the  input.  In 
our  work,  we  rely  only  on  sEMG  since  no  injected 
electrodes  will  be  used  to  obtain  EMG  signals. 
Hence  the  task  is  to  develop  a  model  and  an 
estimation  scheme  for  describing  the  dynamics  of 
the  skeletal  muscle  force  and  finger  angles  from  the 
sEMG  signals.  Some  of  the  recent  efforts  in  this 
direction  are  evident  in  the  research  work  of  [7-13]. 

Present  research  focus  on  the  dynamic  modeling 
and  estimation  of  the  angles  of  the  proximal 
interphalangeal  (PIP)  joint  of  the  index  finger  with 
the  corresponding  sEMG  signal.  An  array  of  nine 
sEMG  sensors  is  used  to  record  sEMG  signals  and 
joint  angles  are  recorded  using  a  wheel 
potentiometer  from  the  arm  of  a  healthy  subject, 
which  respectively  is  explained  in  detail  in  Section  2 
and  3.  Both  the  signals  are  filtered  and  smoothing 
spline  curve  fitted  with  0.993  as  smoothing 
parameter.  The  dynamic  modeling  is  achieved  using 
the  approach  of  System  Identification  (SI)  where 
smoothed  sEMG  data  is  the  input  and  the  smoothed 
joint  angle  data  is  the  output.  Multiple  linear  and 
nonlinear  models  are  obtained.  To  achieve  better 
estimate  of  the  finger  angles,  an  adaptive 
probabilistic  Kullback  Information  Criterion  (KIC) 
for  model  selection  based  data  fusion  algorithm  is 
applied  to  the  linear  and  nonlinear  models  outputs. 
Final  fusion  based  output  of  this  approach  results  in 
improved  estimates  of  finger  angles. 

2  Experimental  Set-Up 

An  experimental  set-up  is  designed  to 
simultaneously  acquire  the  sEMG  from  the  motor 
point  of  the  index  finger  and  the  joint  angle  of  the 
PIP  joint  of  the  index  finger.  Nine  DE-3.1  sEMG 
sensors  of  the  DELSYS®  Bagnoli-16  EMG  system 
are  placed  in  a  three-by-three  array  [14];  the 
arrangement  covers  the  four  directional  spatial 
distributions  of  the  sEMG  signal.  The  middle  three 
sEMG  sensors  were  attached  directly  on  the  skin 
surface  above  the  motor  point  of  the  index  finger  of 
a  healthy  subject.  Using  a  lOk-Ohm  wheel 
potentiometer,  an  angle  measurement  device  is 
designed  to  measure  the  joint  angles  of  the  PIP  joint 
of  the  index  finger  of  a  healthy  subject.  Appropriate 
sEMG  electrode  attachment  point  for  the  motor 
point  of  the  index  finger  was  identified  using  a  wet 


probe  muscle  stimulator  at  the  FDS  (RICH-MAR, 
HV  1000). 


Fig.  1:  Experimental  Set-Up. 


Prior  to  placing  the  sEMG  sensors,  the  skin 
surface  of  the  subject  was  prepared  according  to 
International  Society  of  Electrophysiology  and 
Kinesiology  (ISEK)  protocols.  An  Interlink™ 
Electronics  FSR  0.5”  circular  force  sensor  on  a 
stress  ball  is  used  to  press  with  the  movement  to 
introduce  some  resistance  to  the  movement  of  the 
PIP  joint  of  the  index  finger.  Experimental  set-up  is 
shown  in  Fig.  1  where  nine  sensors  are  shown  on  a 
healthy  subject  forearm,  with  an  angle  measurement 
device  on  the  PIP  joint  of  the  index  finger  and 
having  a  stress  ball  for  force  measurements.  The 
various  signals  are  sampled  at  a  rate  of  2000  Hz 
using  Lab  VIEW™  in  conjunction  with  DELSYS® 
Bagnoli-16  EMG  and  NI  ELVIS™.  With  this 
experimental  set-up  we  conducted  several 
experiments  of  30  seconds,  45  seconds  and  60 
seconds  durations. 

3  Signal  Pre-Processing 

The  recorded  sEMG  signal  for  the  index  finger  is 
filtered  using  nonlinear  spatial  filtering  for  nine 
channel  surface  EMG.  The  nonlinear  spatial  filter  is 
obtained  from  the  “Nonlinear  Teager-Kaiser  Energy 
(TKE)  Operator,”  [14].  Equation  (1)  gives  the  TKE 
operator  in  the  time  domain  on  sEMG  signal  [14]. 

T[x(n)]  =  x2(n )  -  x(n  +  1)  *  x(n  -  1).  (1) 

Here,  T7  is  the  TKE  operator  and  x(n)  is  the  time 
domain  sEMG  signal  at  location  n.  The  four- 
directional  nonlinear  spatial  filter  can  be  derived 
from  the  TKE  operator.  Equation  (2)  gives  the  four- 
directional  nonlinear  spatial  filter. 

4  [A:(m,  n)]  =  4  *  x2(m,ri)  —  x{m  —  l,n)  * 
x(m  +  1,  ri)  —  x(m,  n  —  1)  *  x(m,  n  +  1)  — 
x(m  —  1,  n  +  1)  *  x(m  +  1,  n  —  1)  -  x(m  —  1,  n  — 
1)  *  x(m  +  1,  n  +  1).  (2) 
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Index  Finger  Raw  sEMG  Signal 


Fig.  2:  Raw  and  Spatially  Filtered  sEMG  Signal 
from  Index  Finger. 


Measured  Angle  Theta  -  Index  Finger 


Fig.  3:  Raw  and  Chebyshev  Type  II  Filtered  PIP 
Joint  Angle  for  Index  Finger. 


Fig.  2  shows  a  comparison  between  the  measured 
and  nonlinear  spatial  filtered  sEMG  signals.  A 
Chebyshev  type  II  infinite  impulse  response  (HR) 
low  pass  filter  with  a  550  Hz  pass  band  frequency  is 
used  to  filter  the  PIP  joint  angle  data.  Fig.  3  shows 
the  comparison  between  the  measured  and 
Chebyshev  type  II  filtered  skeletal  muscle  force 
signal. 

4  Smoothing  Spline  Curve  Fitting 

Any  data  can  be  curve  or  surface  fitted  using  the 
MATLAB®  Curve  Fitting  Toolbox™  [15]. 
Smoothing  spline  which  is  a  piecewise  polynomial 
computed  from  a  smoothing  parameter  (p)  of  0.993 
is  fitted  to  the  filtered  sEMG  and  angle  data. 
Smoothing  parameter  (p)  is  a  number  between  0 
and  1 .  Change  in  the  value  of  p  from  0  to  1  results 
in  the  change  in  smoothing  spline.  For  p  =  0  the 
smoothing  spline  is  a  least-square  straight-line 
approximation  to  the  data,  whereas  for  p  =  1  it 
gives  the  "natural”  cubic  spline  interpolant  to  the 
data  [15]. 

For  the  specified  weight  (wt)  and  smoothing 
parameter  (p)  a  smoothing  spline  s  is  formed 
which  minimizes  the  parameter  J  given  by  Equation 
(3). 


J  =  V  Si  wt(yt  -  s(Xi))2  +  (1  -  p)  J (j^)2dx, 

(3) 

where  xt  and  yi  are  predictor  and  response  data 
respectively.  The  smoothing  parameter  p  should  be 
chosen  to  make  the  error  E(s)  =  W;(y;  —  s(x;))2 

d?  s 

and  roughness  J small.  Here  in  this  work 
the  smoothing  parameter  p  is  chosen  as  0.993  [15]. 

5  System  Identification 

System  Identification  (SI)  is  a  tool  to  model  a 
dynamic  system  based  on  input-output  data  [16].  In 
this  research,  we  use  the  SI  to  model  two  signals 
which  are  smoothing  spline  curve  fitted.  The  sEMG 
is  the  input  to  the  system  and  the  intended  PIP  joint 
angle  is  the  output.  Multiple  linear  and  nonlinear 
models  are  obtained  for  modeling  of  sEMG  and  PIP 
joint  angle  signals  for  the  index  finger  of  the 
dominant  hand  of  a  healthy  subject.  Five  linear  and 
three  nonlinear  models  are  obtained  for  the  input 
and  output  data  set.  Table  I  and  II  gives  the 
structures  of  the  linear  and  nonlinear  models 
respectively. 

The  model  order  of  the  various  models  used  in 
this  work  are  as  follows:  linear  models  for  the  input 
and  output  data  set,  OE  model  of  order  16,  ARX 
model  of  order  18,  ARMAX  model  of  model  order 
16,  State-Space  model  with  subspace  method 
(N4SID)  of  order  18  and  a  State-Space  model  with 
prediction  error/maximum  likelihood  method  (PEM) 
of  order  12  are  obtained  using  SI.  Table  I  gives  the 
structure  of  all  the  linear  models.  In  table  I  y  is 
output,  t  is  time,  B(q),  F(q),  A(q),  C(q),  D(q), 
are  polynomials,  q  is  a  backward  shift  operator,  u  is 
input,  nk  is  delay  and  e  is  error  [17].  For  State- 
Space  models:-  x  is  state,  t  is  time,  Ts  is  sampling 
time,  u  is  input,  e  is  error,  A,  B,  K,  C,  and  D  are 
system  matrices,  and  y  is  output  [17]. 


TABLE  I 

LINEAR  MODELS  AND  THEIR  STRUCTURES 


Linear  Model 

Name 

Linear  Model  Structure 

Output  Error 

y(t)  =  j^u(t  -  nk)  +  e(t) 

ARX 

A(q)y(t)  =  B(q)u(t  -  nk )  +  e(t) 

ARMAX 

A(q)y(t)  =  B{q)u(t  -  nk )  + 
C(q)e(t) 

State- Space  - 
subspace  method 

x(t  +  Ts)  =  Ax(t)  +  Bu(t)  +  Ke(t) 

y(t)  =  Cx(t)  +  Du(t)  +  e(t) 

State- Space  - 
prediction 
error/maximum 
likelihood  method 

x(t  +  Ts)  =  Ax(t)  +  Bu(t)  + 
Ke(t) 

y(t)  =  Cx(t)  +  Du(t)  +  e(t) 
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Table  II  gives  the  structures  of  nonlinear  models. 
The  nonlinear  models  for  the  input  and  output  data 
set  are  obtained  as,  the  nonlinear  Wiener- 
Hammerstein  models  with  nonlinearity  estimators  of 
'piecewise  linear  -  pwlinear,’  'sigmoidnet,’  and 
'wavelet  network.’ 

In  Table  II,  for  the  nonlinear  Hammerstein- 
Wiener  model  u(t)  and  y(t)  are  the  inputs  and 
outputs  for  the  system,  respectively.  /  and  h  are 
nonlinear  functions  that  corresponds  to  the  input  and 
output  nonlinearity,  respectively.  w(t)  and  x(t)  are 
internal  variables.  w(t)  has  the  same  dimension  as 
u(t).  x(t)  has  the  same  dimension  as  y(t).  B(q) 
and  F(q)  in  the  linear  dynamic  block  are 
polynomials  in  the  backward  shift  operator.  For 
nonlinear  Hammerstein- Weiner  model  with  wavelet 
network  as  nonlinearity  estimator,  jc(s)  as  a 
wavelet  function,  and  /3k  is  a  row  vector  such  that 
fik(x  —  yk)  is  a  scalar.  If  only  the  input  nonlinearity 
is  present,  the  model  is  called  the  Hammerstein 
model.  If  only  the  output  nonlinearity  is  present,  the 
model  is  called  the  Wiener  model  [17]. 

TABLE  II 


NONLINEAR  MODELS  AND  THEIR  STRUCTURES 


Nonlinear  Model  Name 

Nonlinear  Model  Structure 

Nonlinear  HW  - 
Piecewise  Linear  - 
pwlinear 

w(t)  =  f(u(t )), 
x(t)  =  J  w(t), 
y(t )  =  h  (x(t)); 

Nonlinear  HW  - 
Sigmoidnet 

yv(t)  =  f(y(t-l),y(t- 
2 ),y(t  —  3),  —  l),u(t  — 

2), ...); 

/(*)  =  e-z+ils 

sigmoid  function. 

Nonlinear  HW  - 
Wavelet  Network 

g(x)  =  ££=  i  akK(J]k(x  -  Yk )) 

6  Adaptive  Data  Fusion 

Data  fusion  of  multiple  outputs  of  the  linear  and 
nonlinear  models  is  done  by  assigning  a  particular 
probability  to  each  individual  output  [18].  This 
method  gives  good  estimation  of  the  PIP  joint 
angles.  The  goodness  of  the  fit  of  each  linear  and 
nonlinear  model  output  can  be  computed  using  SI 
model  fit  value,  which  is  given  by  Equation  (4). 

FIT  =  [1-|  Y-Y  |/|  Y  -  7  |  ]  *  100.  (4) 

In  Equation  (4)  Y  is  measured,  Y  is  estimated  and 
F  is  mean  output  signal.  The  model  selection 
criterion  used  in  this  paper  is  Kullback  Information 
Criterion  (KIC)  which  has  shown  to  perform  well 
for  sEMG  sensor  data  fusion  [9-13].  The  sum  of  two 
directed  divergences,  which  is  the  measure  of  the 


models  dissimilarity,  is  known  as  Kullback’ s 
symmetric  or  J-divergence  [19],  as  given  by 
Equation  (5). 


KIC(pi)=\logRi  + 
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where  g(n )  =  n  *  login/ 2). 

The  fusion  algorithm  as  given  by  [18]  is  applied 
for  data  fusion  of  the  outputs  of  the  linear  and 
nonlinear  models  obtained  using  SI: 

(I)  Find  the  linear  and  nonlinear  models  outputs 
Mlf  M2, ... ,  Mk  for  k  number  of  outputs. 

(II)  Compute  the  residual  square  norm 

Ri  =  \\Y-<t>i  0,||2  =  ||r-f  II2, 
where  0;  =  {0^0  and 
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(III)  Calculate  the  model  criteria  coefficient  using 
Equation  (5). 

(IV)  Compute  the  model  probability  p(MjZ)  = 

e-1i 

— ; - r?  where  l  is  model  selection  criterion,  i.e. 


KICipi). 

(V)  Compute  the  fused  model  output  Yf  = 

E&pCMiiz)?!. 

(VI)  Compute  the  overall  model  from  Yf  and 
skeletal  muscle  force  data. 

In  this  algorithm  the  computation  from  step  (II) 
to  (VI)  is  adaptive  i.e.  the  residual  square  norm, 
KlC(Pi),  model  probability  p(Mi|Z),  and  fused 
model  output  Yf  are  being  updated  with  time  or  for 
each  data  point. 


7  Results  and  Discussion 

The  sEMG  signals  from  the  sensor  array  and  angle 
signals  of  PIP  joint  of  index  finger  are  filtered; 
smoothing  spline  curve  fitted  and  modeled  using 
multiple  linear  and  nonlinear  models.  First  the 
outputs  of  linear  and  nonlinear  models  are  fused 
separately  and  then  the  resultant  outputs  are  fused 
using  an  adaptive  KIC  based  probability.  This 
approach  gives  improved  estimates  of  the  finger 
angles  of  the  PIP  joint  of  the  index  finger  of  the 
dominant  hand  of  a  healthy  subject.  Results  are 
presented  in  the  following  figures.  Fig.  4  shows  the 
overlapping  plot  of  the  measured-curve  fitted  angle 
vs.  data  fusion  based  angle  using  linear  models.  Fig. 
5  shows  the  overlapping  plot  of  the  measured-curve 
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fitted  angle  vs.  data  fusion  based  angle  using 
nonlinear  models. 


Curve  Fitted  Angle  Vs.  Data  Fusion  Angle  Using  Linear  Models 


Fig.  4:  Curve  Fitted  Vs.  Data  Fusion  Based  Angle 
Using  Linear  Models. 


Curve  Fitted  Angle  Vs.  Data  Fusion  UsingNonlinear  Models 


Fig.  5:  Curve  Fitted  Vs.  Data  Fusion  Based  Angle 
Using  Nonlinear  Models. 


Curve  Fitted  Angle  Vs.  Data  Fusion  Using  Linear  -  NL  Models 


Fig.  6:  Curve  Fitted  Vs.  Data  Fusion  Based  Angle 
Using  Linear-Nonlinear  Models. 

Fig.  6  shows  the  overlapping  plot  of  the  curve 
fitted  angle  vs.  data  fusion  based  angle  using  linear- 
nonlinear  models.  It  is  clearly  evident  that  the  data 
fusion  based  output  follows  the  measured-curve 
fitted  output.  Mean  percentage  error  of  the  linear, 
nonlinear  and  linear-nonlinear  models  data  fusion 
based  angles  are  2.5191e-005,  -4.5807e-006,  and 
8.1 167e-006  respectively. 

Fig.  7  shows  the  validation  plot  for  a  different 
experimental  data  where  the  measured-curve  fitted 
and  the  linear-nonlinear  modeled  data  fusion  based 
angle  signal  shows  a  close  follow  up.  Mean 


percentage  error  of  the  linear,  nonlinear  and  linear- 
nonlinear  models  data  fusion  based  angles  are 
2.8202e-005,  3.3191e-005,  and  6.8079e-006 

respectively. 


Validation  Plot  1  -  Curve  Fitted  Angle  Vs.  Data  Fusion  Based  Angle 


Fig.  7:  Validation  Plot  -  Curve  Fitted  Vs.  Data 
Fusion  Based  Angle  Using  Linear-Nonlinear 
Models. 


8  Conclusion  and  Future  Work 

The  dynamic  modeling  of  the  filtered  and  smoothed 
sEMG  and  PIP  joint  angle  of  the  index  finger  is 
achieved  using  SI  with  sEMG  is  the  input  and  the 
joint  angle  is  the  output.  Multiple  linear  and 
nonlinear  models  are  obtained  for  the  input  and 
output  data.  To  achieve  a  better  estimate  of  the 
finger  angles,  an  adaptive  probabilistic  Kullback 
Information  Criterion  (KIC)  for  model  selection 
based  data  fusion  algorithm  is  applied  to  the  linear 
and  nonlinear  models  outputs.  Final  fusion  based 
output  of  this  approach  results  in  improved 
estimates  of  finger  joint  angles.  The  approach  is 
validated  using  a  different  set  of  experimental  data. 

As  this  initial  study  shows  potential  in  the  pursuit 
of  controlling  an  artificial  hand  on  an  instantaneous 
basis,  we  will  further  this  work  in  the  future  by 
improving  the  data  collection  techniques  and 
optimizing  the  experimental  procedure  as  well  as 
optimizing  the  smoothing  parameter  for  the 
smoothing  spline  curve  fitting. 
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Abstract 

Genetic  Algorithm  (GA)  are  biologically  motivated 
evolutionary  computation  techniques  used  both  as 
search  methods  for  solving  problems  and  for 
modelling  evolutionary  systems.  GA  has  been  used  in  a 
wide  variety  of  optimization  tasks,  including  numerical 
optimization  and  combinatorial  optimization  problems 
such  as  circuit  design  and  job  shop  scheduling. 
Although  there  are  many  problems  for  which  the 
genetic  algorithm  can  evolve  into  a  good  solution  in 
reasonable  time,  there  are  also  problems  for  which  it 
is  inappropriate  like  the  one  in  which  it  is  important  to 
find  the  exact  global  optimum.  In  addition,  if  the 
number  of  iteration  increases  the  algorithm  will  take  a 
long  time  to  complete  its  computations.  In  this  work, 
we  are  using  GA  to  optimize  the  parameters  of 
nonlinear  Bayesian  Filters  for  electromyographic 
(EMG)  signal.  OpenMP  is  a  parallel  programming 
model  for  shared  memory  and  distributed  shared 
memory  multiprocessors.  We  are  using  OpenMP  on 
four  processors  to  improve  the  time  of  GA  optimization 
for  Bayesian  Filter  parameters. 

Keywords:  OpenMP,  Matlab ®,  MPI,  EMG,  GA, 
Gene,  Mother  Chromosome,  Father  Chromosome, 
Pairing  Operator,  Mating  Operator,  Mutation, 
Population  Size. 

1.  Introduction 

Principles  of  natural  selection  and  genetics  has  been 
used  to  deduce  some  efficient  search  algorithms,  GA  is 
one  of  them.  GA  is  used  in  different  fields  to  obtain 
solutions  to  different  problems,  e.g.  engineering, 


science,  business  and  economics  [1,  2].  Some  of  the 
applications  of  GA  are  in  automotive  design, 
engineering  design,  robotics,  evolvable  hardware, 
optimized  telecommunication  routing,  joke  and  pun 
generation,  bio-mimetic  invention,  traffic  and 
shipment  routing,  computer  gaming,  encryption  and 
code  breaking,  computer-aided  molecular  design,  gene 
expression  profiling,  optimizing  chemical  kinetic 
analysis,  finance  and  investment  strategies,  and 
marketing  and  merchandising.  Although,  for  a  general 
problem  of  normal  size,  GA  is  a  good  option  to  get  the 
optimal  solution  in  appropriate  amount  of  time, 
whereas  for  bigger  and  complex  problems  GA  takes 
more  time  to  find  optimal  solution.  Researchers  have 
explored  different  approaches  to  minimize  this 
computational  time  and  cost  and  parallel  processing 
came  out  to  be  one  of  the  most  promising  approaches. 

GA  is  based  on  the  principle  of  natural  selection.  It 
uses  a  random  search  technique  to  find  the  optimal 
solution  of  a  problem  by  manipulating  a  population  of 
candidate  solutions.  To  form  next  generation,  GA 
selects  the  best  solutions  from  the  evaluated 
population.  Good  solutions  and  time  to  reach  them  by 
GA  depends  on  the  population  size  [3,  4],  because  if 
the  population  is  too  small,  then  the  search  space  will 
not  be  sufficient  explored  and  it  will  be  difficult  to 
identify  good  solutions.  In  the  case  when  the 
population  is  too  big,  the  GA  will  waste  computational 
resources  processing  unnecessary  individuals.  Each 
individual  in  the  population  has  a  fitness  value,  which 
is  a  payoff  measure  that  depends  on  how  well  the 
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individual  solves  the  problem.  Crossover  and  mutation 
are  the  two  operators  used  by  GA  to  explore  the  search 
space,  here  crossover  is  a  primary  and  mutation  is  a 
secondary  search  operator.  The  probability  of 
crossover  is  much  higher  than  the  probability  of 
mutation.  GA  can  be  stopped  based  on  different 
conditions,  one  is  by  using  a  predetermined  number  of 
generations  and  function  evaluations,  second  is  the 
average  quality  of  the  population  gets  saturated  after 
some  number  of  generations,  and  the  third  is  when  all 
the  individuals  are  identical,  which  can  only  occur 
when  mutation  is  not  used.  We  are  using 
predetermined  number  of  generations  and  function 
evaluations  to  stop  the  GA  in  this  research  work.  The 
generic  GA  flow  chart  is  shown  in  Figure  1. 


Figure  1 :  Generic  GA  Flow  Chart. 


The  time  taken  by  GA  is  directly  proportional  to  the 
number  of  evaluations  and  generations,  the  more  the 
number  of  evaluations  the  lager  is  the  computational 
time  for  GA.  On  the  other  hand,  this  time  can  be 
minimized  by  paralleling  the  GA  program  on  a  parallel 
cluster.  Most  of  the  parallel  programs  divide  the  task  in 
chunks  and  solve  it  simultaneously  using  multiple 
processors.  On  a  parallel  cluster,  there  are  numerous 
processors,  and  they  share  the  task,  the  larger  the 
number  of  processors  the  faster  the  computation  or 
lesser  the  time  for  the  completion  of  the  algorithm.  In 
this  research  work  we  are  using  OpenMP  parallel 
programming  model  and  utilizing  a  cluster  of  four 
processors  to  run  the  GA  and  comparing  its 
performance  with  the  single  processor  in  Matlab®. 


2.  Background 

Most  of  the  parallel  programs  divide  the  task  into 
chunks  and  solve  it  simultaneously  using  multiple 
processors.  Similar  approach  can  be  used  for  GAs  in 
different  ways.  These  approaches  can  be  using  a  single 
population,  divide  the  population  into  several 
subpopulations,  use  of  massively  parallel  architecture, 
or  use  of  fewer  multicomputer  with  fewer  and  more 
powerful  processing  units.  Mainly  there  are  three  kinds 
of  parallel  GAs,  and  these  are  (a)  global  single¬ 
population  master-slave  GAs,  (b)  single-population 
fine  grained,  and  (c)  multiple-population  coarse¬ 
grained  GAs.  The  hierarchical  parallel  GAs  combine 
the  multiple  demes  with  master-slave  or  fine-grained 
GAs.  It  has  a  higher  level  and  a  lower  level,  at  the 
higher  level  they  are  multiple-deme  algorithms  and  at 
the  lower  level  they  are  single-population  parallel  GAs 
(either  master-slave  or  fine-grained)  [1]. 

The  research  work  in  [5]  shows  a  method  of  parallel 
GA,  in  which  a  global  parallel  implementation  of  the 
conventional  GA  and  a  GA  with  a  generation  gap, 
where  some  part  of  the  population  is  replaced.  The 
efficiency  of  global  parallel  GA  is  close  to  100  %  in 
SIMD  (single  instruction  multiple  data)  computers  by 

[5] .  Four  prototypes  for  parallel  GAs  are  proposed  in 

[6]  where  the  first  three  are  variations  of  the  master- 
slave  GA,  and  the  fourth  is  a  multiple-population  GA. 

2.1  Master-Slave  Parallelization 

Same  as  in  a  simple  GA,  master-slave  GA  also  have 
a  single  panmictic  population.  Figure  2  shows  the 
schematic  of  master-slave  GA,  where  the  evaluation  of 
fitness  is  distributed  among  the  slave  processors. 


Figure  2:  Master- Slave  Parallel  GA. 
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The  master-slave  GA  considers  full  population  size 
for  selection  and  crossover,  so  it  is  known  as  global 
GA.  The  master-slave  parallelization  method  for  GA 
does  not  change  the  behavior  of  the  algorithm  while 
the  single-population  fine  grained  and  multiple- 
population  coarse-grained  GA  change  the  way  the  GA 
works  [1].  The  master-slave  parallel  GA  considers  all 
the  population  for  selection,  whereas  the  other  parallel 
GAs  considers  only  a  subset  of  the  population.  In 
addition,  in  the  master-slave  parallel  GA  the  mating  is 
random,  whereas  in  the  other  methods  mating  is 
restricted  to  a  subset  of  individuals  [1].  The  majority  of 
the  global  parallel  GAs  are  implemented 
synchronously.  If  the  multiprocessor  architecture  is  of 
shared  memory  then  each  individual  processor  can 
share  the  information,  reading  and  writing  without  any 
issue.  In  contrast,  on  distributed-memory  architecture, 
the  population  is  stored  only  at  one  processor  and  the 
master  is  responsible  for  sending  the  individuals  to 
slaves  for  evaluation,  collecting  the  results  and 
applying  GA  for  next  generation.  Encore  Multimax  is 
a  shared-memory  computer  with  16  processors,  a  GA 
is  implemented  on  this  to  search  for  efficient 
timetables  for  schools  in  [7].  This  research  in  [7]  gave 
a  limited  speed-up  because  of  the  critical  path  of  the 
program  serial  code. 

In  1993,  Abramson,  Mills,  and  Perkins  added  a 
distributed-memory  machine  “Fujitsu  API 000”  with 
128  processors  to  the  experiments,  changed  the 
application  to  train  timetables  and  modified  the  code 
[8].  In  this  research,  the  speed-up  was  significant  up  to 
16  processors  and  then  decreased  because  of  the 
increase  in  communication  as  more  processors  are 
added. 

Hauser  and  Manner  used  global  GA  on  three 
different  parallel  processors  and  got  good  speed  up 
only  on  NERV  multiprocessor  [9].  The  speed-up  of 
five  using  six  processors  is  obtained  with  a  very  low 
communications  overhead  [9].  The  reason  of  low 
performance  on  other  processors  a  SparcServer  and  a 
KSR1  is  because  of  the  inadequate  scheduling  of 
computation  threads  to  processors  by  the  system.  All 
the  previous  research  results  shows  that  the  master- 
slave  parallel  GAs  are  easy  to  implement  and  very 
efficient  method  of  parallelization  especially  when  the 
evaluation  needs  considerable  computations. 
Additionally  the  master-slave  parallel  GAs  do  not 
change  the  search  behavior  of  the  GA  [1]. 


3.  GA  In  Matlab® 

The  continuous  number  GA,  which  is  used  in  this 
work,  interprets  the  parameters  of  a  problem  as  genes, 
where  one  set  of  parameters  form  a  chromosome, 
chromosome  =  [genelf  gene2, ,  geneNpar  }.  Since 
one  chromosome  corresponds  to  a  candidate  solution, 
its  evaluation  is  done  by  assessing  the  numerical  value 
of  that  solution,  which  usually  referred  as  the  cost. 
Unlike  traditional  optimization,  the  function  is  not 
referred  to  as  the  cost  function  but  as  the  fitness 
function  or  objective  function.  Using  an  initial 
population  size  with  randomly  generated 

chromosomes,  where  each  gene  is  generated  by  a 
random  number  generator  within  the  search  space,  the 
chromosome  will  take  the  dimension  of 

RNpar  population  size  .  After  evaluating  each 

chromosome  cost  ct  =  f  {chromosome  t),i  = 
1,2, population  size,  the  selection  operator  will 
select  a  subset  of  the  chromosomes  based  on  their  cost 
value  to  survive  for  the  next  iteration. 

For  elitism  based  GA,  the  best  performing 
chromosomes  always  survive.  There  are  two  options 
the  population  size,  a  steady-state  or  a  performance 
based  size.  The  first  option  maintains  a  certain  number 
of  chromosomes  for  each  iteration  -  also  referred  to  as 
a  generation;  while  performance  based  size,  selection 
sets  a  minimum  criterion  for  chromosomes  to  survive. 
The  latter  option  may  cause  the  algorithm  to  end 
prematurely  since  not  minimum  criteria  (cost  value). 

Pairing  Operator:  Genetic  Algorithm  may  employ 
a  number  of  pairing  mechanisms.  Among  them  are 
pairing  from  top  to  bottom,  where  the  best  performing 
chromosome  is  paired  with  the  next  best  performing 
chromosome  and  so  on;  random  pairing,  where  a 
uniform  distribution  is  assumed  to  randomly  pick  the 
mother  and  father  chromosome;  weighted  random 
pairing  where  the  probability  of  selection  is  influenced 
by  the  performance  of  the  individual  chromosome;  and 
rank  weighing,  where  the  probability  of  selection  is 
influenced  by  the  ranking  of  chromosome  with  respect 
to  the  others  measured  again  by  the  performance.  In 
this  paper,  the  pairing  is  accomplished  by  determining 
the  cumulative  probability  for  each  chromosome  is 

P(ri)  =  n  .,  where 
U=i 1 

_  population  size  —no  .of  chromosomes  kept 

r  ~  2  ’ 
and  n  is  the  index  of  chromosomes.  Having  computed 
the  individual  cumulative  probabilities  P(n),  the 
parent  chromosomes  are  now  selected  by  utilizing  a 
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random  number  generator  and  indexing  this  with  the 
cumulative  probability.  Hence,  the  best  performing 
chromosome  has  the  best  chance  of  being  selected,  but 
also,  all  chromosomes  in  the  mating  pool  have  a 
chance  of  being  selected. 

Mating  Operator:  In  this  work,  the  next  generation 
chromosomes  are  created  from  the  mother  and  father 
chromosome  (which  were  selected  as  described  above) 
by  following  equations  given  as: 

p6Ttenew  i 

mother(a)  —  (3  (mother  (a)  —  father(a )), 
genenew  2 

father  (a)  +  (3  (mother  (a)  —  father  (a)), 

Where,  a  =  integer  (rand  *  Npar  ), 

(3  =  random  number  between  0  —  1,  and  Npar  is 
number  of  parameters  for  complete  solution.  In  this  a 
can  also  be  viewed  as  equivalent  to  the  cross-over 
point  utilized  in  the  binary  GA.  Hence,  the  offspring 
(new  chromosomes)  are  found  as 

chromnewi  = 

(g^nemoi jieri, genemot  jier2, ... ,  genenew  lf ..., 

denedadNpar  )> 

and 

chromneW2  (gcvi€dadi> 9enedad2>  •••  > 9enenew2>  ■■■' 

9enemotherNpar  ). 

Mutation:  Mutation  is  an  important  operator  that 
allows  the  search  to  escape  local  minima  in  order  to 
find  better  areas  on  cost  surface.  Too  much  mutation 
will  do  opposite,  i.e.  if  a  chromosome  approach  an 
optimum  point,  it  might  mutate  and  be  placed  at  an 
entirely  different  location  of  the  cost  surface.  Mutation 
is  accomplished  by  randomly  changing  a  gene,  i.e. 
substitute  it  with  a  random  numbers. 


4.  GA  In  OpenMP 

OpenMP  is  a  parallel  programming  model,  which 
uses  some  compiler  directives  and  library  routines  to 
express  shared-memory  parallelism  [10].  OpenMP 
uses  Fortran  and  C++  interfaces,  which  is  a  standard 
feature.  OpenMP  API  is  designed  and  developed  by  a 
group,  which  represents  the  major  vendors  of  high- 
performance  computing  hardware  and  software.  A  set 
of  compiler  directives  is  the  major  part  of  OpenMP 
interface.  The  programmer  adds  these  compiler 
directives  to  a  sequential  program.  The  programmer 


specifies  the  synchronization  points  and  tells  the 
compiler  what  parts  of  the  program  to  execute 
concurrently.  As  the  directives  can  be  added  in  a 
incremental  fashion,  the  OpenMP  gives  a  path  and 
platform  for  the  parallelization  of  existing  software.  In 
contrary,  the  Pthreads  and  MPI  approaches  are  library 
routines.  Pthreads  and  MPI  are  linked  with  and  called 
from  a  sequential  program,  and  the  programmer  is 
required  to  divide  the  computational  work  manually 
[11]. 

Literature  does  not  show  any  evidence  of  the  use  of 
OpenMP  parallel  programming  model  for  GA  in 
specific.  This  work  is  using  OpenMP  to  optimize  the 
filter  parameters  of  Bayesian  filters  using  GA 
technique.  The  parallel  cluster  has  four  processors.  The 
idea  of  the  research  is  to  implement  the  GA  using 
OpenMP  on  this  four  processors  cluster  and  compare 
the  results  with  the  output  on  Matlab®  on  a  single 
processor.  The  OpenMP  parallel  programming  model 
is  given  in  Figure  3,  where  sequential  parts  and  parallel 
regions  are  labelled  clearly. 


Sequential  Part 

Parallel  Region 

Sequential  Part 

Parallel  Region 

Sequential  Part 


Figure  3:  OpenMP  Parallel  Programming  Model. 

5.  Results  and  Discussions 

In  this  work,  we  are  using  GA  to  optimize  the 
parameters  of  nonlinear  Bayesian  Filters  for 
electromyographic  (EMG)  signals.  OpenMP  is  a 
parallel  programming  model  for  shared  memory  and 
distributed  shared  memory  multiprocessors.  We  are 
using  OpenMP  on  four  processors  to  improve  the  time 
of  GA  optimization  for  Bayesian  Filter  parameters. 
First  Matlab®  is  used  to  run  GA  on  one  processor, 
and  the  same  algorithm  is  applied  on  OpenMP  parallel 
processing  model  using  four  processors.  Two  sets  of 
data  captured  at  a  sampling  rate  of  2000Hz  are  used, 
first  data  is  for  30  seconds  and  the  second  is  for  60 
seconds.  The  total  simulation  time  taken  by  the 
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Matlab®  and  OpenMP  is  recorded  precisely. 
Following  tables  shows  the  results  obtained  from 
simulations.  Table  1,  2  and  3  shows  the  Matlab® 
simulation  results  for  two  different  data  sets  of 
different  lengths. 


Table  1:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab®. 


No.  of 

Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 

set. 

Simulation  time 
seconds  -  60 
seconds  of  data 

set. 

10,000 

7.13 

2393.06 

15,000 

16.95 

2403.52 

20,000 

28.05 

2416.66 

25,000 

42.66 

2432.89 

30,000 

60.61 

2452.34 

35,000 

82.41 

2475.94 

40,000 

108.21 

2503.69 

45,000 

139.35 

2535.91 

50,000 

175.45 

2571.72 

Table  2:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab®. 


No.  of 
Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 
set. 

Simulation  time 
seconds  -  60 
seconds  of  data 
set. 

60,000 

208.65 

2613.84 

70,000 

260.67 

2741.73 

80,000 

331.85 

2813.34 

90,000 

428.03 

2913.25 

100,000 

550.89 

3039.70 

Table  3:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab®. 


No.  of 
Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 
set. 

Simulation  time 
seconds  -  60 
seconds  of  data 
set. 

125,000 

704.54 

3193.36 

150,000 

943.89 

3433.06 

175,000 

1288.28 

3775.16 

200,000 

1755.77 

4236.28 

Figures  1,  2  and  3  are  graphical  representation  of  the 
results  from  the  Table  1,  2  and  3  respectively.  It  is 
evident  from  Figure  1,  2  and  3  that  CPU  time  is 
directly  proportional  to  the  data  size  and  number  of 
iterations.  Therefore,  increase  in  the  data  size  and  the 
number  of  iterations  results  in  the  increase  in  CPU 
time. 


Iteration  Vs.  CPU  Time  (Seconds)  ■  30  Seconds  Data  Set  (Matlab) 


Figure  1:  Iterations  Vs.  CPU  Time  -  Table  1  Data. 


Iteration  Vs.  CPU  Time  (Seconds)  ■  30  Seconds  Data  Set  (Matlab) 
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Figure  2:  Iterations  Vs.  CPU  Time  -  Table  2  Data. 


Iteration  Vs.  CPU  Time  (Seconds)  -  30  Seconds  Data  Set  (Matlab) 


Figure  3:  Iterations  Vs.  CPU  Time  -  Table  3  Data. 

The  GA  code  from  Matlab®  is  converted  into  C- 
code  and  OpenMP  implementation  is  under  pipeline. 
Results  will  be  published  in  next  reputed  conferences. 
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The  C-code  can  be  found  at 
http://www.isu.edu/-chiustev/research/FRC964.html. 

6.  Summary  and  Future  Work 

To  conclude  this  work  uses  the  idea  of  parallel 
processing  and  implement  GA  on  OpenMP  parallel 
programming  model  of  four  processors.  First  Matlab® 
GA  code  is  converted  to  C  code  and  then  to  OpenMP 
to  implement  on  the  parallel  cluster.  The  algorithm  is 
implemented  both  in  OpenMP  and  Matlab®  for  25, 
50,  75  and  100  iterations  and  the  time  taken  in 
execution  is  recorded.  As  expected  the  results  shows 
the  improvement  of  time  in  OpenMP  implementation. 
In  future  work,  similar  approach  will  be  applied  using 
MPI  and  OpenMP  and  MPI  combined  for  GA  and 
more  complex  and  advanced  optimization  algorithms. 
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Abstract:  -  This  paper  presents  a  short  study  on  the  hybridization  of  a  swarm  based  optimization  algorithm 
with  a  single  agent  based  algorithm.  Swarm  based  algorithms  and  single  agent  based  algorithms  have  each 
distinct  advantages  and  disadvantages.  One  goal  of  the  presented  work  is  to  combine  the  concepts  of  the  two 
different  algorithms  such  that  a  more  effective  optimization  routine  results.  In  particular,  we  used  a  Particle 
Swarm  (PS)  based  optimization  algorithm  as  basis  and  induce  Tabu  Search  (TS)  based  operators.  The 
developed  hybrid  algorithm  is  tailored  such  that  has  the  capability  to  adapt  to  the  given  cost  function  during 
the  optimization  process.  The  proposed  algorithm  is  tested  on  a  set  of  different  benchmark  problems.  In 
addition,  the  hybrid  algorithm  is  utilized  for  solving  the  estimation  problem  encountered  when  colored  noise  is 
present  and  the  Least  Squares  (LS)  algorithm  has  bias  problems. 

Key -Words:  -  Particle  Swarm,  Optimization,  Tabu  Search,  Parameter  Estimation,  Colored  Noise 


1  Introduction 

Meta-heuristic  approaches  such  as  Particle  Swarm 
Optimization  (PSO)  and  Tabu  Search  (TS) 
algorithms  have  been  around  for  a  few  years.  They 
present  alternative  approaches  to  find  optimality  for 
a  diverse  set  of  optimization  problems,  such  as 
estimation  and  system  identification,  among  others. 
PSO  and  TS  based  approaches  stand  in  contrast  to 
hard  computing  methods  such  as  gradient  based 
methods  by  vastly  reducing  the  computational 
complexity,  especially  for  higher  dimensional 
problems.  A  key  motivation  for  using  these  soft- 
computing  algorithms  is  the  ability  to  escape  local 
optimal  points  in  pursuit  of  the  global  optimum 
during  the  search  process. 

PSO  was  first  introduced  by  Kennedy  and 
Eberhart  as  a  stochastic  global  optimization  routine, 
[1].  The  algorithm  searches  the  cost-surface  with  a 
set  of  individual  particles,  each  representing  a 
solution  to  the  optimization  problem.  The  particles 
orient  themselves  via  influence  components  such  as 
inertia,  current  best  solution  found  by  the  entire 
flock,  and  the  particle’s  best  position.  TS  was 
presented  by  Glover,  [2],  as  an  iterative  optimization 
procedure  that  attempts  to  avoid  the  shortfalls  of  the 
Local  Search  (LS)  algorithm.  TS  is  a  single  agent 
algorithm  that  progresses  through  the  cost  surface  by 
creating  a  set  of  neighbors  (which  are  potential  new 
locations)  and  moves  to  a  new  location  by 
evaluating  these  neighbors  and  comparing  them  to  a 


list  of  previously  visited  locations.  This  list,  the  tabu 
list,  ensures  that  no  cycling  will  occur  and  local 
optimal  points  are  overcome. 

Recent  efforts  in  improving  PSO  and  TS  have 
lead  to  a  number  of  propositions  that  include 
concepts  from  other  optimization  philosophies.  In 
[3]  the  authors  proposed  a  TS-Genetic  Algorithm 
based  scheme  to  find  best  parameter  estimates  in 
colored  noise  environments.  Higashitani  et  al.,  [4] 
proposed  a  predator-prey  based  PSO.  The 
Constriction  Factor  Method  (CFM)  proposed  by 
Clerc  [5]  reduces  the  velocity  of  the  particles  with 
duration  of  the  search  process  in  order  to  facilitate 
the  intensification  over  the  diversification  mission 
with  time.  Combinations  of  PSO  and  TS  have  also 
been  investigated,  such  as  in  [6],  where  PSO  and  TS 
are  switched  at  each  step  to  explore  the  vicinity  of 
the  particles,  and  in  [7]  where  TS  and  PSO  and 
sequential  quadratic  programming  are  combined.  In 
[8]  concepts  of  TS  are  used  to  create  two  swarms  in 
PSO,  one  responsible  for  intensification,  the  other 
for  diversification. 

In  this  paper,  we  investigate  various  elements  of 
the  TS  algorithm  applied  to  the  PSO  scheme.  In 
particular  we  investigate  an  alternative  to  the  CFM 
method,  were  we  use  the  TS  concept  to  adapt  the 
PSO  parameters  to  the  given  problem  using  a 
parallel  scheme  of  hybridization.  The  paper  is 
organized  as  follows;  Section  2  presents  the  problem 
formulation  and  basic  background  on  PSO  and  TS. 
Section  3  introduces  the  proposed  PSO-TS 
hybridization  algorithm,  while  Section  4  will  present 
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the  results.  Section  5  entails  some  conclusions  based 
on  the  presented  simulation  results  and  prior  works. 


2  Problem  Formulation 


The  optimization  problems  addressed  in  this  paper 
are  of  the  unconstrained  type.  Consider  a  real  valued 
objective  function  /(x)  defined  on  a  set  xgS  in 
W1 .  We  are  tasked  to  find  a  point  x  corresponding 
to  the  value  of  f(x)  such  that 

/(x*)  =  min{/(x)|x  e  S]  (1) 

The  difficulty  of  finding  such  a  point  arises  when 
f[x)  or  the  feasible  domain  S  is  non-convex.  For 
such  problems,  gradient  based  methods  will  result  in 
local  optimums.  Soft  computing  methods  such  as 
PSO  and  TS  allow  for  an  escape  from  local 
optimums  and  the  continued  search  for  a  better 
optimum,  in  the  extreme  case  the  global  optimum. 
For  PSO  algorithms,  we  consider  a  set  of  particles 
p  and  their  associated  location  x*  where  i  is  the 
index  for  the  particles.  The  particles  explore  S  in  an 
incremental  fashion.  Their  position  is  altered  at  each 
step  by  the  following  update  rule: 

x'  (k  +  l)  =  x  (£)  +  v'  (£  +  l),  (2) 

where}  k  is  the  increment  number  and  v*  the 


velocity  associated  with  the  particle,  which  can  be 
computed  as  follows: 

(*+ 1)  =  wv'(A;)  +  Cj/i  [b;  -  X1'  (*)]  + 

+c2r2  [bg -x‘ (&)]. 

In  the  velocity  update  formula,  and  b  represent 
the  best  ever  position  of  the  particle  i  ancf  the  global 
best  position  of  the  swarm  up  to  iteration  k , 
respectively.  rx  and  r2  are  uniformly  distributed 
random  numbers  between  0  and  1,  while  w,  c7  and  c2 
are  weighting  coefficients  that  manage  the  three 
tendencies  of  Equation  (3).  These  tendencies  as 
given  by  the  three  terms  in  the  velocity  equation  are 
characterized  as:  audacious  for  following  your  own 
way;  conservative,  for  going  back  towards  your  best 
previous  position;  and  sheep-like,  for  being  pulled  to 
the  best  overall  position.  The  weighting  coefficients 
help  the  balancing  act  of  combining  these  tendencies 
in  order  to  be  globally  efficient.  The  PSO  scheme 
described  so  far  is  rather  simple  and 
computationally  very  efficient,  especially  when 
compared  to  a  GA  or  any  gradient  based  methods. 
To  further  this  algorithm,  a  limitation  on  the 
maximum  value  of  V  can  be  imposed,  say  vmax. 
This  type  of  velocity  clamping  balances  the 
exploration  and  intensification  and  also  avoids 
excessive  velocities  that  are  responsible  for  particles 
overshooting  their  targets,  [9].  Also,  the  CFM 
method  can  be  employed,  where  the  velocity  update 
is  modified  -  reduced  -  as  the  search  progresses.  For 
this,  Equation  (3)  is  altered  as  follows: 

V(k  + 1)  =  K  x  [v- (k)  +  c/x  (V  -  x'  (£))]  + 

(4) 

i:x[+c2r2(b;-x'V;))] 


where 


K  = 


and  (p  =  cl  +  c2 ,  as 


given  by  [5]. 

The  inertia  weight  w  has  a  competing  purpose  to 
the  velocity  clamping  method  [10],  and  is 
responsible  for  balancing  the  exploration  and 
intensification  part  of  the  PSO.  In  [11]  it  is  also 
shown  that  w  is  problem  dependent.  In  this  paper, 
we  will  investigate  the  option  of  adapting  w  using  a 
TS  based  scheme  such  that  the  PSO  algorithm 
becomes  less  problem  dependent.  Besides  w,  cx 
and  c2  control  the  stochastic  influence,  analogous 
to  the  mutation  operator  in  GA.  In  [12],  it  is  stated 
that  for  most  problems  one  chooses  cx  -  c2 ,  but  the 
ratio  is  also  problem  dependent,  i.e.  for  unimodal 
problems  with  smooth  surfaces,  usually  one  selects 
c2  >cx,  while  for  multimodal  problems  these 
coefficients  are  chosen  to  be  ci  > •  In  this  paper, 
we  also  will  investigate  the  optimization  of  these 
coefficients  by  implementing  a  TS  scheme  in 
parallel  to  the  PSO. 

The  TS  algorithm  has  the  same  objective  as  PSO- 
given  by  Equation  1 .  A  TS  algorithm  employs  one 
candidate  point  which  moves  incrementally  on  S . 
The  moves  between  increments  are  defined  by 
selecting  the  best  update  from  a  set  of  candidate 
points.  This  set  is  created  by  forming  R  n- 
dimensional  spheres  about  the  current  position,  and 
randomly  place  a  point  in  each  section  (sphere). 
These  points  are  evaluated  on  /(x)  and  then 
compared  to  a  list  T  e  Wxq  of  previously  visited 
places.  The  list  length  q  basically  represents  the 
memory  of  the  algorithm.  Around  each  element  in 
T  a  ft -dimensional  sphere  is  created  serving  as  a 
tabu-ball,  where  no  new  solution  can  be  located.  The 
best  performing  candidate  point  not  violating  the 
restricted  spaces  given  by  the  q  tabu  balls  is 
selected  as  the  new  position  of  the  search.  An 
aspiration  criterion  can  be  formulated  as  an  escape 
clause  to  circumvent  the  restrictions  imposed  by  the 
tabu-balls,  often  in  the  form  of  having  an  improved 
cost  to  the  current  solution.  All  of  the  past  best 
solutions  are  stored  in  a  new  list  -  the  promising  list, 
which  is  assembled  as  the  TS  algorithm  processes. 
The  optimum  of  the  entries  of  that  list  will  be  used 
as  the  final  optimal  value  of  the  algorithm. 

Usually,  q ,  R ,  and  the  dimension  of  the 
individual  R' s  are  considered  the  control 
parameters  of  the  TS  algorithm,  which  are  all 
problem  dependent. 


3  Problem  Solution 

The  selection  of  w ,  cx ,  and  c2  have  rather  large 
implication  on  the  update  of  the  new  velocity  vector 
v'(*  +  l),  as  is  illustrated  in  Fig.  1.  Here,  the  three 
influence  vectors  with  regard  to  the  global  best 
solution,  -x  and  with  regard  to  the  particles 
best  solution  b  -x  [k)  as  well  as  the  inertia  vector 
v'  (&)  are  depicted  for  an  instant  k  and  a  single 
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particle  i.  Each  of  these  vectors  are  pre-multiplied 
by  w,  cx ,  and  c2  respectively,  resulting  into  some 
variation  of  the  vectors,  denoted  by  A  in  Fig.  1. 


Location  of 


Fig.  1:  Adaptation  of  proposed  PSO-TS  Algorithm. 

As  stated  earlier,  these  three  influence  coefficients 
are  problem  dependent.  In  this  paper,  we  propose  to 
find  the  optimal  coefficients  by  utilizing  a  TS.  This 
is  accomplished  by  using  the  variation  of  the 
influence  vectors  to  describe  a  new  search  area,  in 
which  the  original  cost  function  /(x)  is  prescribed. 
In  Fig.  1,  this  area  is  symbolized  in  the  shaded 
ellipsoid.  The  best  combination  of  coefficient  along 
with  the  influence  vectors  will  then  determine  the 
velocity  update  v'(&  +  l).  This  optimization  is 
repeated  at  each  increment.  TS  lends  itself  well  for 
this  purpose,  since  it  is  computationally  very 
efficient.  Since  r7  and  r2  are  superimposed  to  the 
influence  coefficients,  we  will  investigate  an  option 
where  this  randomization  is  suppressed. 

Parameter  estimation  is  an  important  concept  in 
engineering  where  a  mathematical  model  of  a 
system  is  identified  with  the  help  of  input  and  output 
signals.  The  classical  Least  Squares  (LS)  algorithm 
gives  an  unbiased  estimate  of  the  parameters  when 
the  system  noise  is  white.  A  bias  result  may  occur  if 
the  residual  s(k)  =  y(k)  -  y(k)  is  not  white,  where 
y  is  the  system  output,  y  is  the  estimated  output, 
and  k  is  the  discrete  time  index.  For  such  instances, 
one  can  use  a  whitening  filter,  i.e.  C(q~l)e(k)  =  e{k) , 
where  C(z_1)  is  the  coefficient  polynomial  of  the 
whitening  filter  and  z”1  is  the  backward  shift 
operator.  The  corresponding  cost  function  for  the 
estimation  becomes,  [3]: 

J  =  ^e\k)  =  Y,\.C{z')A{z')y{k)  +  C{z)B{z')u{k)-\^ 
where,  ku{k)  is  the  input  sequence,  A(zl)  and  B(z~l ) 
are  the  numerator  and  denominator  polynomials 
of  the  system  transfer  function,  respectively.  As 
the  signal  to  noise  ratio  becomes  large,  the  cost 
function  J  may  become  multimodal.  This  was 
shown  by  Soderstrom  et  al.  [13].  In  order  to 
obtain  the  global  minimum  an  intelligent 
optimization  technique  needs  to  be  employed.  In 
this  paper,  we  will  utilize  the  proposed  hybrid  PSO- 
TS  algorithm  to  such  parameter  estimation  problems 


with  a  colored  noise  environment  and  compare  it  to 
the  traditional  LS  method. 


4  Simulation  Results 

The  proposed  hybrid  algorithm  is  tested  on  a  set  of 
standard  optimization  benchmark  problems,  which 
are  listed  below: 

n 

Spherical:  /(x)  =  ^.w  and 

M 

n  f  7 

Quadratic:  /(x)  =  X  2>, 

j= i  Ui 

where  x.  e  [-100,100]  and  /(V)  =  [0,0]. 

n 

Hyperellipsoid:  /(x)  =  ^J2x2j  where  x.  e[-l,l] 

7=1 

and  /(x* )  =  [0,0] . 

Bohachevsky  1: 

f(x)  =  x]+2x\-03  cos  (3;^  )  -  0.4  cos  (4;rx2  )  +  0.7 
where  x.  e  [-50, 50]  and  / (V  )  =  [0, 0] . 


Ras trigin:  /(x  )=IF2-10  cos(2;rx.)  +  10) 

7=1 

where  x.  e  [-5.12,5.12]  and  /(V)  =  [0,0]. 
Rosenbrock: 


nil  r-  ? 

/(x)  =  I  100  (x1]-x2lj_l)  +(l--V,) 


where 


7=1 

x.  e  [—2.048, 2.048 J  and  /(x*)  =  [  1, l] . 

Haupt  &  Haupt:  f  (x)  =  xl  sin (4Xj )  +  l.lx2  sin(2x2 ), 
where  x.  e  [0,10]  and  /(x*)  =  [9.0389,8.6674] . 


In  addition  to  these  benchmark  problems,  we  also 
will  test  the  proposed  algorithm  to  some  parameter 
estimation  problem  with  a  colored  noise 
environment.  In  particular,  we  will  test  the  algorithm 
on  one  simple  parametric  model,  which  are  given  as 
follows: 

y(^)  =  0.5y(^-l)-0.5y(^-2)  +  w(^)  +  ^(^)  (4  ) 

The  simulation  was  carried  out  by  using  the 
following  set  of  parameters  and  conditions: 
uniformly  distributed  initial  locations  of  particles, 
number  of  neighborhoods  created  at  each  step  for 
each  particle  R  =  25 ,  uniform  initial  velocity  of 
magnitude  1  for  all  particles,  p  =  40 ,  q  =  5 ,  radius 
of  the  n  dimensional  Tabu  balls: 
r  =  0.025  (xmax  -xmin)  ,  where  xmx  and  x  in  are  the 
dimensions  nof  S  ” radius  of  neighborhood  =  0.5r  , 
and  radius  of  promising  balls  rp  =  0.5 rN.  The  PSO 
portion  utilized  100  iterations,  while  the  embedded 
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TS  section  used  10  iterations.  Since  randomness  is  a 
factor  in  the  used  and  proposed  algorithms,  and  in 
order  to  gain  some  kind  of  understanding  of 
consistency,  the  simulations  were  carried  out  100 
times  for  each  cost  function  listed  above. 

Table  1  lists  the  results  for  the  unconstraint 
optimization  problems  in  comparison  with  the 
original  PSO  algorithm.  In  this  set  of  simulations,  r7 
and  r2  were  selected  to  be  uniform  random  between 
0  and  1 . 


Table  1:  Simulation  results 


Cost  Function 

PSO 

Hybrid  PSOTS 

w 

w,  cl 

W,C!C2 

Spherical 

0.55135 

0.17214 

0.0555 

0.2216 

Quadratic 

0.50718 

0.25637 

0.12497 

0.1347 

Bohachevsky 

0.80852 

0.17689 

0.15473 

0.01617 

Hyperellipsoid 

0.03252 

0.00436 

0.00745 

0.00069 

Rastrigin 

0.07168 

0.16678 

0.16728 

0.16021 

Rosenbrock 

1.4261 

1.43255 

1.47011 

1.44042 

Haupt 

12.4736 

10.5563 

10.5329 

11.5222 

Table  2  lists  the  results  for  the  unconstraint 
optimization  problems  in  comparison  with  the 
original  PSO  algorithm.  In  this  set  of  simulations,  r7 
and  r2  were  set  equal,  at  all  times,  to  1.  It  is 
interesting  to  note  that  for  almost  all  the  cases,  the 
proposed  hybrid  optimization  algorithm,  regardless 
of  which  set  of  coefficients  are  optimized,  performs 
significantly  better  in  comparison  to  the  standard 
PSO  for  the  Hyperellipsoid,  the  Bohachevsky  cost 
function.  For  this  case  too,  the  spherical  and  the 
quadratic  functions  produced  better  results.  In  both 
the  cases  there  was  no  improvement  in  the 
Rosenbrock  function  values  and  the  Rastrigin  still 
showed  poor  results  as  compared  to  the  standard 
PSO  algorithm. 

The  distinction  between  the  characteristics  of  the 
cost  function  and  the  performance  of  the  proposed 
algorithm  can  be  made  by  associating  the  presented 
results  with  unimodal  and  multimodal  cost 
functions.  As  listed  above,  the  improved 
performance  given  by  the  proposed  hybrid  algorithm 
is  found  for  unimodal  cost  functions.  While  the 
proposed  algorithm  does  work  well  for  multimodal 
cost  functions,  it  does  excel  in  comparison  to  the 
standard  PSO  algorithm  with  regard  to  unimodal 
functions. 


Table  2:  Simulation  resu 

Its 

Cost  Function 

PSO 

Hybrid  PSOTS 

w 

w,  cl 

W,CiC2 

Spherical 

0.55135 

0.08975 

0.19801 

0.81325 

Quadratic 

0.50718 

0.31023 

0.14124 

0.07529 

Bohachevsky 

0.80852 

0.13091 

0.20773 

0.15158 

Hyperellipsoid 

0.03252 

0.01464 

0.01736 

0.00278 

Rastrigin 

0.07168 

0.13648 

0.23206 

0.4902 

Rosenbrock 

1.4261 

1.44026 

1.44147 

1.39044 

Haupt 

12.4736 

10.6342 

10.8366 

10.8408 

The  hybrid  algorithm  was  also  tested  to  estimate  the 
parameters  of  the  system  given  by  equation  (4).  The 
estimated  system  parameters  are  given  in  Table  3 
for  the  two  cases  a)  Random  r7  and  r2  and  b)  Fixed 
r7  and  r2  to  unity.  In  addition,  we  distinguished  two 
cases  where  we  either  computed  the  instantaneous 
optimum  value  of  the  influence  coefficients  and  the 
accumulated  “averaged”  optimum.  The  variance  in 
the  noise  was  set  at  1%,  while  the  input  sequence 
was  a  white  noise  sequence  with  variance  of  30%. 

Table  3:  Simulation  Results  for  Estimation 


Random  | 

Accumulative 

w 

0.5898 

-0.4583 

1.0425 

w,cl 

0.5319 

-0.3717 

1.1217 

w,cl,c2 

0.4704 

-0.4666 

0.8921 

Just  Optimum 

w 

0.5340 

-0.5159 

1.0025 

w,cl 

0.5339 

-0.5234 

1.0550 

w,cl,c2 

0.4820 

-0.4622 

1.0110 

Fixed  [ 

Accumulative 

w 

0.5289 

-0.3794 

1.0061 

w,cl 

0.5018 

-0.3799 

1.0777 

w,cl,c2 

0.478 

-0.4917 

0.9182 

Just  Optimum 

w 

0.4707 

-0.4827 

1.0515 

w,cl 

0.4797 

-0.4397 

1.0260 

w,cl,c2 

0.5074 

-0.5164 

0.9794 

The  hybrid  algorithm  gave  the  best  estimated 
values  of  the  system  parameters  for  the  following 
combination;  Fixed  r7  and  r2,  just  optimum,  w,  c7,  c2 
optimized,  and  40  particles.  Table  3  is  giving  the 
average  values  based  on  10  sample  points.  This 
combination  was  then  used  to  compare  the  results  of 
the  hybrid  algorithm  with  Least  Squares  (LS).  Table 
4  shows  the  comparison  between  LS  and  the 
proposed  hybrid  algorithm  for  various  variances  in 
the  noise  level.  The  hybrid  PSOTS  again 
outperforms  the  LS  method  in  almost  all  the  cases 
and  in  some  estimated  values.  These  cases  are 
highlighted  in  bold  italics.  Simulation  runs,  for  the 
hybrid  algorithm,  for  variances  0.05,  0.09,  0.1  and 
0.2  were  increased  from  10  to  30  which  gave 
slightly  better  results  once  again  in  comparison  to 
the  LS. 

A  key  objective  of  this  paper  is  to  make  the 
proposed  algorithm  adaptive  in  nature  to  the 
subjected  problem,  i.e.  cost  function.  As  detailed  in 
Section  2,  the  choice  of  the  influence  coefficients  is 
dependent  on  the  characteristics  of  the  cost  surface. 
The  proposed  hybrid  algorithm  optimizes  these 
coefficients  as  well  as  the  inertia  coefficient  w  at 
each  iteration.  Hence  we  can  plot  these  coefficients 
in  order  to  determine  if  they  show  adaptation  and 
consistency.  This  is  done  in  Fig.  2.  The  plot  is  a 
representative  characterization  of  the  simulation 
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results  obtained  for  the  tested  multimodal  cost 
functions.  We  can  recognize  that  the  proposed 
algorithm  adapts  fast  to  the  cost  function  and  selects 
cx  >  c2  as  determined  in  the  literature,  indicating 
that  each  particle  is  dominated  by  its  own  personal 
best,  and  the  algorithm  is  searching  wider  ranges. 
The  rather  non-smooth  convergence  plot  is  due  to 
the  fact  that  r7  and  r2  are  kept  random  and  impose  a 
perturbation  to  the  optimal  location  of  the  influence 
coefficients.  In  comparison,  the  same  plots  are 
generated  for  some  multimodal  cost  functions,  i.e. 
the  Hyperellipsoid  in  Fig.  3  and  Spherical  as 
depicted  in  Fig.  4.  In  both  cases  for  the  unimodal 
cost  functions,  the  coefficients  move  quickly  to  the 
case  where  c2  >  cx  as  given  in  the  literature,  but 
after  a  few  iterations,  the  weighting  is  switched  and 

Cl  >  C2 


Table  4:  Comparison  Hybrid  PSOTS  and  LS 


Simulation  Options:  Optimized  w,cl,c2,  Fixed  r1  and  r2 
and  Just  Optimum,  Simulation  Runs  :  10 

Hybrid  PSOTS 

Least  Squares 

Variance:  0.001 

[0.5265,-0.4688,0.9695] 

[-0.00640,-0.4238,0.8794] 

Variance:  0.005 

[0.5256,-0.5206,1.0531] 

[0.0005,-0.5050,0.8703] 

Variance:  0.009 

[0.5289,-0.4394,0.9640] 

[0.0640,-0.5395,0.8626] 

Variance:  0.01 

[0.5074,-0.5164,0.9794] 

[0.0208,-0.4711,0.8592] 

Variance:  0.05 

[0.4183,-039%%, 0.9539] 

[0.1354, -0.5345, 0.8100] 

Variance:  0.09 

[0.3300,-02699,0.9840] 

[0.0981, -0.7102, 0.9573] 

Variance:  0.1 

[0.4035,-03056,1.0322] 

[0. 1276, -0.66M, 0.8821] 

Variance:  0.2 

[0.2952, -0.1684, 0.9861] 

[0.04 18,-0. 7096,0.9323] 

Variance:  0.05,  Simulation  runs: 30 

[0.4476,-0.3844,1.0049] 

[0.1354, -0.5345, 0.8100] 

Variance:  0.09,  Simulation  runs: 30 

[0.3852,-0.3505,0.9672] 

[0.0981,-0. 7102,0.9573] 

Variance:  0.1,  Simulation  runs: 30 

[0.4129,-0.3550,0.9922] 

[0.1276, -0.6684, 0.%%2l] 

Variance:  0.2,  Simulation  runs:30 

[0.3072,-0.2769,0.9970] 

[0.0418, -0.7096, 0.9323] 

This  implies  that  the  particles  are  no  longer 
dominated  by  the  global  optimum,  but  rather  by 
their  own  personal  best  performance  and  try  to 
improve  their  performances.  An  explanation  for  this 
is  found  by  considering  the  convergence  plot,  as 
detailed  in  Fig.  5.  Here  we  can  see  that  most  of  the 
convergence  is  achieved  after  the  first  few  iterations. 
Once  not  much  improvement  is  found,  the  algorithm 
tries  to  spread  out  and  search  larger  regions  rather 
than  concentrating  to  the  area  dominated  by  the 


current  global  best  solution.  This  adaptation  may 
explain  the  cause  for  the  improved  performance  of 
the  proposed  algorithm  for  unimodal  functions 
compared  to  standard  PSO  algorithms. 
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Fig.  2:  Plot  of  optimized  influence  coefficients. 
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Fig.  3  :  Plot  of  optimized  influence  coefficients  for 
Hyperellopsoid  cost  function. 
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Fig.  4:  Plot  of  optimized  influence  coefficients  for 
the  Spherical  cost  function. 
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Fig.  5:  Plot  of  global  best  cost  vs.  iteration  index  for 
the  Spherical  cost  function. 

4  Conclusion 

In  this  paper,  we  propose  a  hybrid  PSO-TS 
algorithm  for  optimization  of  unconstraint  cost 
functions.  The  algorithm  adapts  to  the  underlying 
cost  surface  by  adjusting  the  influence  coefficients 
at  each  step.  The  proposed  algorithm  is  tested  on  a 
series  of  benchmark  problems  and  showed  improved 
performance  for  unimodal  cost  functions.  In 
addition,  the  hybrid  PSO-TS  algorithm  is  tested  on 
two  parameter  estimation  problems  where  colored 
noise  influence  causes  bias  estimates  when  the  LS 
algorithm  is  used.  The  simulations  indicate  an 
improved  estimate  when  the  proposed  PSO-TS 
algorithm  is  used. 
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ABSTRACT 

Surface  Electromyographic  (sEMG)  signals  have  been 
exploited  for  almost  a  century,  for  various  clinical  and 
engineering  applications.  One  of  the  most  compelling  and 
altruistic  applications  being,  control  of  prosthetic  devices.  The 
study  conducted  here  looks  at  the  modeling  of  the  force  and 
sEMG  signals,  using  nonlinear  Hammers tein-  Weiner  System 
Identification  techniques.  An  extensive  study  has  been  conducted 
on  the  modeling  of  sEMG  and  corresponding  force  data  to 
establish  a  relation  which  can  mimic  the  actual  force 
characteristics  for  a  few  particular  hand  motions.  Analysis  of  the 
sEMG  signals,  obtained  from  specific  Motor  Unit  locations 
corresponding  to  the  index,  middle  and  ring  finger,  and  the  force 
data  led  to  the  following  deductions;  a)  Each  motor  unit  location 
has  to  be  treated  as  a  separate  system,  (i.e.  extrapolation  of 
models  for  different fingers  cannot  be  done)  b)  Fatigue  influences 
the  Hammerstein-Wiener  model  parameters  and  any  control 
algorithm  for  implementing  the  force  regimen  will  have  to  be 
adaptive  in  nature  to  compensate  for  the  changes  in  the  sEMG 
signal  and  c)The  results  also  manifest  the  importance  of  the 
design  of  the  experiments  that  need  to  be  adopted  to 
comprehensively  model  sEMG  and  force. 

NOMENCLATURE 

Bjfs),  Fj  fs)  Polynomials  in  the  linear  block. 

bif  Ci  Coefficients  of  polynomials  (Hammerstein  Model) . 

fn  Objective  function  (PSO). 

gf)  Input  nonlinearity  function  (Hammerstein  Model). 


g(')  Static  nonlinear  function. 
h()  Output  nonlinearity  function. 

n  b  Number  of  zeros  plus  one. 

n  f  Number  of  poles. 

n  k  Delay  from  the  input  to  the  output  in  terms  of  the  number 
of  samples. 

u(t)  Input  to  a  system. 

v(t),  w(t)  Internal  variables. 

x(t)  Input  nonlinearity  (Hammerstein  Model). 

y(t)  Output  of  a  system. 

y(t)  Predicted  or  estimated  output. 

z(t)  Output  linear  block  (Wiener  Model). 

xj  (t)  The  ith  component  position  of  the  jth  particle  in  time  t. 

x/’ibestf)  The  local  best  position  of  the  jth  particle  in  generation  t. 

xj,gbest (^  jjie  gX0bal  best  position  of  the  jth  particle  in  generation 
t. 

v/  (t)  The  ith  component  velocity  of  the  jth  particle  in  time  t. 
t  time. 

H  Physical  parameters  of  non-linear  function  g(*). 
a  Expected  rate  of  gradual  drift. 

f  Expected  rate  of  sudden  shifts  in  surface  EMG. 

s  Tolerance. 
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0  Finite  dimensional  vector  used  to  parameterize  the 
mapping. 

INTRODUCTION 

For  centuries  man  has  been  fascinated  by  the  human  body 
and  it’s  working.  A  great  number  of  scientists,  doctors  and 
engineers  have  spent  a  great  deal  of  time  and  effort  interpreting 
various  biological  signals  and  decoding  the  information  contained 
within  them.  Though  the  task  at  hand  was  very  complicated,  it 
never  deterred  researchers  from  undertaking  this  challenge  with 
great  determination,  which  led  to  some  ground  breaking  work  in 
this  field.  But  inspite  of  all  the  advances  made  in  these  fields,  we 
still  oftentimes  find  ourselves  coming  up  short  in  understanding 
the  complexities  of  the  human  body.  One  such  field  is 
rehabilitation  engineering.  The  biological  signal  of  interest  here  is 
the  Surface  Electromyogram  signal  (sEMG).  We  have  come  a 
long  way  since  the  first  ever  study  conducted  on  EMG  signals  by 
H.  Piper  in  1912,  [1].  About  50%  of  the  people  who  need  upper 
extremity  prosthetics  don’t  use  them,  [2]  which  puts  into 
perspective  the  direction  that  research  on  in  this  area  needs  to 
focus  on.  The  ideal  scenario  would  be  to  mimic  the  real  human 
hand  with  all  its  functionality  and  dexterity  probably  using  EMG 
signals.  This  goal  seems  to  be  quite  distant  as  of  today. 

EMG  signals  have  been  used  for  quite  some  time  now  in  the 
control  of  prosthetic  devices.  Simply  put,  it  is  analogous  to  having 
small  voltage  signals,  with  coded  information  of  the  specific 
desired  objective,  to  actuate  motors  which  would  translate  this 
information  into  the  desired  force  dynamics. 

EMG  signals  are  fairly  complex  to  work  with.  In  order  to  have  a 
standardized  method,  on  how  EMG  signals  must  be  recorded  and 
analyzed,  the  International  Society  of  Electromyography  and 
Kinesiology  (ISEK)  has  a  set  of  guidelines  which  are  widely 
accepted.  These  guidelines  for  reporting  EMG  data  are 
documented  in,  [3].  Other  signal  analysis  techniques  generally 
employed  include  low-pass  or  band-pass  filtering;  others  have 
also  employed  notch  filtering  to  remove  power  line  noise,  [4]. 
Whitening  filters  have  also  been  employed  in  order  to  increase  the 
quality  of  the  amplitude  estimates  of  the  surface  EMG  signals, 
[5].  Some  of  the  other  signal  processing  methods  include  using 
Markov  models,  [6]  and  fuzzy  logic  control,  [7]  -  for 
classification  of  EMG,  wavelet  processing,  [8],  and  Bayesian 
estimators,  [9].  Extensive  research  has  also  been  carried  out  in 
obtaining  a  relationship  between  sEMG  and  force  -  for  example, 
linear  and  non-linear  sEMG-Force  by  Wood  et  al .,  [10],  EMG- 
Force  relationships  in  skeletal  muscles  by  Jacquelin  et  al.,  [11], 
where  a  linear  relationship  was  approximately  quantified  for  the 
middle  operating  force  range  during  an  isometric  effort.  They  also 
concluded  that  the  presence  of  any  movement,  the  algebraic 
relations  estimated  were  not  sufficient  to  describe  the  relation. 
They  also  concluded  that  considerable  attention  needs  to  be  given 
to  the  data  acquisition,  the  type  of  electrode,  amplification, 
rectification,  sampling,  noise  filtering  etc.  as  all  these  elements  do 
have  an  impact  on  the  mathematical  formulation  of  the  sEMG- 
Force  relation.  The  effects  of  joint  position,  gravity  and 


recruitment  have  been  studied  by  Solomonow  et  al.  in  [12].  The 
effects  of  fatigue  on  EMG-Force  relation  was  investigated  by 
Moxham  et  al  [13].  Studies  investigating  the  fatigue  of  motor 
units  in  muscles  that  control  the  fingers  of  the  hand  were  carried 
out  by  Fuglevand  et  al.  This  study  showed  that  fatigue  caused  the 
force-frequency  relation  to  be  displaced  to  higher  frequencies. 

The  work  reported  in  this  paper  is  focused  on  using  non-linear 
System  Identification  techniques,  to  come  up  with  a  suitable 
model  or  models  for  the  sEMG-Force  relation.  Also  modeling  for 
the  effects  of  fatigue  on  the  sEMG-Force  relation  is  taken  into 
consideration  and  a  guideline  is  provided  on  the  type  of 
experiments  that  would  be  most  beneficial  to  encompass  the  large 
number  of  variables  which  affects  the  sEMG  signal  over  time. 
The  results  of  a  number  of  nonlinear  models  used  for  deducing 
the  sEMG-Force  relationship  strongly  indicate  the  need  for  an 
adaptive  method  to  be  developed  to  capture  the  dynamics  of  the 
two  signals.  The  results  also  indicate  that  the  modeling  of  the 
sEMG-Force  relation  should  not  be  extrapolated  to  different 
fingers  even  though  the  experiments  performed  were  almost  the 
same  to  obtain  very  similar  force  signal  patterns.  Bayesian  filters 
were  also  used  in  analyzing  the  sEMG  signals  as  they  have  been 
shown  to  be  beneficial  in  system  identification  [14].  Particle 
Swarm  optimization  was  used  to  assist  in  identifying  possible 
trends  in  the  Bayesian  filter  parameters  with  the  onset  of  fatigue. 

THEORETICAL  BACKGROUND 

EMG,  VARIABLES  AFFECTING  sEMG-FORCE  RELATIONSHIP 
&  PREVIOUS  RESEARCH :  An  EMG  signal  is  an  electrical  signal 
which  is  generated  in  a  muscle  contraction  and  is  representative 
of  neuromuscular  activity.  sEMG  signals  are  measured  at  motor 
unit  location.  A  motor  unit  (MU)  consists  of  an  a  -motoneuron  in 
the  spinal  cord  and  the  muscle  fiber  it  innervates.  Many  factors 
influence  sEMG  signals,  some  of  which  are;  a)  shape  of  the 
volume  conductor,  b)  the  thickness  of  the  subcutaneous  tissue 
layers,  c)  tissue  inhomogeneities,  d)  distribution  of  the  motor  unit 
territories  in  the  muscle,  e)  size  of  the  motor  unit  territories,  f) 
distribution  and  the  number  of  fibers  in  the  motor  unit  territory,  g) 
length  of  the  fibers,  h)  spread  of  the  endplates  and  tendon 
junctions  within  the  motor  units,  and  i)  spread  of  the  innervations 
zones  and  tendon  regions  among  motor  units.  The  type  of 
detection  system  used  also  plays  an  important  part  in  influencing 
the  sEMG  measurements.  Some  of  the  factors  which  need  to  be 
taken  into  account,  with  the  detection  systems,  are  a)  skin 
electrode  contact  (impedance,  noise),  b)  spatial  filtering  for  signal 
detection,  c)  inter-electrode  distance,  d)  electrode  size  and  shape, 
and  e)  inclination  of  the  detection  system  relative  to  the  muscle 
fiber  orientation  [15]. 

Some  of  the  more  recent  studies  suggest  poor  statistical  trends 
between  EMG  amplitude  and  mean  power  spectral  frequency, 
which  indicate  limitations  in  the  use  of  EMG  descriptors  for 
fatigue  assessment  [16].  Mortiz  et  al.  found  evidence  which 
supports  the  hypothesis  that  the  EMG  discharge  rate  variability  is 
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a  major  determinant  of  the  trends  in  isometric  force  variability 
across  the  working  range  of  a  muscle  [17].  Studies  also  suggest 
that  there  is  not  a  significant  difference  between  the  EMG-Force 
relations  obtained  using  intramuscular  EMG  (iEMG)  as  compared 
to  sEMG  [18].  Extensive  modeling  for  the  simulating  sEMG 
signals  detected  by  surface  electrodes  has  also  been  performed 
[19].  For  the  study  conducted  here,  the  sEMG  signals  were 
filtered  using  a  Chebyshev  Type  II  high  pass  filter  and  a 
Chebyshev  Type  II  low  pass  filter.  The  sEMG  signals  were  also 
filtered  using  a  Bayesian  Exponential,  and  a  Half-Gaussian  filter 
similar  to  the  ones  described  in  [20]  and  because  it  has  previously 
been  reported  in  [14]  to  yield  better  fits  for  non-linear 
Hammerstein-Wiener  modeling. 

Particle  Swarm  Optimization :  Over  the  past  few  decades  several 
algorithms  have  been  used  increasingly  to  find  global  minimum 
values  in  an  optimization  problem.  Few  of  the  well  established 
optimization  algorithms  are;  a)  Genetic  Algorithm  (GA)  - 
inspired  by  biological  evolution,  is  at  its  core,  survival  of  the 
fittest.  The  main  drawback  of  GA  being  that  it  is  computationally 
very  expensive  especially  for  large  search  spaces.  GA  has  been 
paired  with  many  other  search  methods  in  order  to  improve  its 
overall  performance,  b)  Tabu  Search  -  which  has  a  set  of  tabu 
balls  and  promising  balls  or  a  tabu  list  which  helps  it  to  reduce  the 
search  space,  and  is  not  very  computationally  expensive.  Some  of 
the  other  optimization  algorithms  are,  Steepest  Descent  Method, 
Gradient  Search  Method,  Ant  Colony  optimization,  Simulated 
Annealing,  Hill  Climbing,  Bees  Algorithm,  Quantum  Annealing 
etc.  In  1995,  a  new  evolutionary  algorithm  was  developed  by  J. 
Kennedy,  a  social  psychologist,  and  R.  Eberhart,  known  as 
Particle  Swarm  Optimization  (PSO).  This  method  is 
computationally  less  expensive  and  has  been  used  in  this  paper  to 
optimize  the  Bayesian  filter  parameters  a  and  f.  Particle  swarm 
optimization  has  a  simple  procedure;  1)  Defining  the  input 
parameters:  This  includes  the  swarm  size  (number  of  particles  of 
the  swarm),  maximum  number  of  iterations,  the  limiting  velocity 
of  the  particles,  the  upper  and  lower  bounds  of  the  search  space 
and  the  stopping  tolerance.  2)  Initializing  the  position  of  the 
particles  -  can  be  randomly  assigned  using  a  uniform  or  normal 
distribution  [21].  3)  Evaluating  the  fitness  function  fn  (objective 
function)  for  the  jth  particle.  This  depends  on  the  cost  function, 
e.g.  the  local  best  and  the  global  best  values  are  compared  and 
suitably  updated.  Moreover,  the  difference 
between  \fn  (t)  -  fn  (t  - 1)| ,  i.e.  the  current  (Qand(f-l)is 
compared  to  the  tolerance  £  .  If  J  fn  (t)  -  fn  (t  - 1)|  <  s  ,  then  stop 
the  loop;  otherwise  continue.  Figures  1  and  2  show  the  particles 
of  the  PSO  in  the  search  space  and  the  flow  chart  of  the  algorithm 
respectively. 

System  Identification  (SI),  Nonlinear  SI,  Hammerstein-Wiener 
Models: 

The  main  aspect  of  SI  involves  modeling,  which  is,  developing  a 
concept  or  mathematical  relation  on  how  the  system  variables  are 
related.  If  a  model  is  constructed  from  observed  data  then  it 
results  into  an  estimation  problem.  The  effort  that  needs  to  go 
behind  modeling  systems  usually  depends  on  the  end  application. 
Acceptable  models  are  those  which  are  defined  by  their 
“usefulness”  rather  than  their  “truthfulness”.  This  means  how 
closely  the  computed  model  is,  in  satisfactorily  representing  the 


true  system.  Modeling  is  divided  into  three  components:  1) 
Accurately  measuring  a  data  set,  e.g.  the  input  and  output  of  a 
system  related  in  the  time  or  the  frequency  domain. 


FIGURE  1:  PREVIOUS  AND  UPDATED  POSITON  OF  PARTICLES  [21] 


FIGURE  2:  PARTICLE  SWARM  OPTIMIZATION  LOOP 

2)  Defining  a  model  structure  and  3)  A  rule  by  which  a  model 
structure  can  be  established  from  the  measured  data,  such  as 
Least- Squares.  Looking  closely  at  these  components,  the 
measured  data  may  be  recorded  during  a  specifically  designed 
experiment,  so  that  the  data  carries  maximum  information,  subject 
to  any  constraints,  [22].  Choice  of  a  model  structure  is  one  of  the 
most  important  and  difficult  parts  in  system  identification.  Here  a 
priori  knowledge  and  engineering  intuition  coupled  with  formal 
properties  of  the  system  will  prove  to  be  invaluable.  Models  of 
systems  with  some  unknown  physical  parameters  can  be 
constructed  from  basic  physical  laws  and  well  established 
relations.  These  models  are  known  as  white  box  models.  Models 
with  unknown  physical  parameters  and  a  large  number  of  random 
variables,  which  cannot  be  satisfactorily  represented  by  physical 
laws,  are  deduced  using  black  box  modeling.  The  third  type  of 
model  is  a  gray  box  model;  this  represents  a  compromise  and  a 
combination  between  white  and  gray  box  modeling.  The  third 
component  of  modeling  is  assessment  of  the  model  quality  and  its 
ability  to  reproduce  the  measured  data,  also  known  as  model 
validation.  The  process  of  System  Identification  is  summarized  in 
Figure  3.  Modeling  and  identification  of  nonlinear  systems  is 
especially  challenging,  in  the  sense  that  the  nonlinearity  could  be 
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in  either  the  input,  the  output  or  in  both.  If  modeling  of  the  system 
is  done  using  linear  and/or  non-linear  blocks,  to  represent  a 
system,  careful  combination  of  these  needs  to  be  formulated.  In 
our  study,  the  input  is  the  sEMG  and  the  output  is  the 
corresponding  force  signal  generated  by  the  human  hand.  The 
system  is  the  particular  subject’s  hand.  The  sEMG  signal  is 
affected  by  large  number  of  variables,  some  of  which  have  been 
mentioned  in  the  EMG  section  of  the  theoretical  background.  The 
force  signals  too  are  non-linear,  especially  as  time  progresses  and 
the  muscle  fatigues.  It  would  be  very  difficult  and  complicated  to 
take  into  account  all  the  factors  which  plague  these  two  signals. 

50  with  the  assumption  that  the  two  signals  are  non-linear;  the 
modeling  for  this  system  can  be  approached  in  a  general  sense.  In 

51  the  model  structure  that  has  been  used  previously  for  capturing 
the  non-linear  input  and  output  system  dynamics  has  been  the 
Hammerstein-Wiener  model. 

Prior 

Knowledge 


FIGURE  3:  SYSTEM  INDENTIFICATION  LOOP  [22] 

Hammerstein  models  assume  a  separation  between  the  non¬ 
linearity  and  the  dynamics  of  the  process  [23].  The  Hammerstein 
structure  consists  of  a  static  non-linear  block  followed  by  a  linear 
dynamic  block.  It  is  described  by  the  following  equations; 

x(t)  =  g(u(t)), 

y(t )  =  b±x(t  - 1)  + . . .  +  bmx(t  —  m)  —  ayy(t  - 1)  - ...  -  amy(t  -  m). 

Hammerstein  models  are  popular  in  the  field  of  control 
engineering  as  it  is  easy  to  compensate  the  non-linear  process 
behavior  by  a  controller  that  implements  the  inverse  static 
nonlinearity  g_1(«)  at  its  output  (if  the  inverse  exists).  Another 
advantage  of  this  model  being  that  the  stability  is  determined 
solely  by  the  linear  part  of  the  model.  Any  assumption  of  the 
structure  of  the  process  is  very  restrictive  and  can  be  applied  to 
only  a  limited  class  of  systems.  The  static  nonlinearity  is  usually 
approximated  using  a  polynomial.  For  systems  with  multiple 


inputs  the  static  nonlinearity  g(«)  becomes  a  higher  dimensional 
function,  say  for  /  =  2 ,  and  m  -  2  Equation  (1)  becomes; 

2 

x(t)  =  cq+c \u(t)  +  c 2U  ( t ), 

y(t)  =  b\CQ+b\c^u(t  - 1)  +  b2C Q+b2C\u(t  -  2)  +  (2) 

2  2 
b^c 2U  (t -1)  +  2U 

The  generalized  Hammerstein  model  is  constructed  by 
summarizing  identical  terms  and  re-parameterizing  the  model  in  a 
manner  that  yields  linear  parameters,  for  example,  Z^and 
b2c0  both  represent  offsets  and  can  be  summarized.  This  leads  to 
Equation  (3); 

y(t)  =  6\  +  Q^uit  - 1)  +  6yu{t  -  2)  + 

(3) 

0Ay(t  - 1)  +  65y(t  -  2)  +  d6u 2  (t  - 1)  +  fyu2  ( t  -  2). 

The  Wiener  model  structure  is  the  Hammerstein  model  reversed. 
The  linear  dynamic  block  is  followed  by  a  static  nonlinear  block. 
This  yields  the  equation  given  in  (4) 

x(t)  =  byu(t  - 1)  + ...  +  bmu(t  -  m)  -  a\x(t  - 1)  -  ...ayxit  -  m),  ... 

y(t)  =  g(x(t)).  ^ 

Eliminating  x  in  (4)  yields; 

yif)  =  g(b\u(t  - 1  )  +  ...  +  bmu(t-m) 

-  a\ g~l  (y(t  - 1))  - ...  -  amg~X ( y(t  -  m)) 

The  combination  of  the  two  is  then  the  Hammerstein-Wiener 
model,  [24]  as  given  in  Figure  4(a)  Hammerstein  b)  Wiener 
models  and  Figure  5  Hammerstein-Wiener  model.  Consider  the 
Hammerstein  case  where  the  static  nonlinear  function  g(*)  can  be 
parameterized  either  in  terms  of  physical  parameters,  such  as 
saturation  point  and  saturation  level,  or  in  black-box  terms  such 
as  spline-function  coefficients.  This  defines  g(*,//) .  If  the  linear 
model  is  given  by  G(q ,  9) ,  the  predicted  output  model  will  be  in 
the  following  form: 

y(t\0,tl)  =  G{q,0)g(u{t),Tj)  .  (6) 

Equation  7  describes  the  Hammerstein-Wiener  model  structure: 
B:Aq) 

Mt)  =  f((g(t)),  HO  =  w(0,  y(0  =  h(x(t)),  (7) 

fjM 

where,  w(t)  and  b(t)  are  internal  variables,  w{t)  has  the  same 
dimensions  as  u{t ) ,  and  x(t )  has  the  same  dimensions  as  y{t) . 


(b) 


FIGURE  4  (A):  A)  HAMMERSTEIN  MODEL,  B)  WIENER  MODEL 
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FIGURE  4  (B):  HAMMERSTEIN-WIENER  MODEL  STRUCTURE 

The  linear  block  is  specified  using  the  terms  nb  -  the  number  of 
zeros  plus  one,  -  the  number  of  poles  and  nk  -  the  delay  from 
the  input  to  the  output  in  terms  of  the  number  of  samples.  The 
commonly  used  nonlinear  estimators  for  Hammerstein-Wiener 
model  are,  a)  Dead  Zone,  b)  Piecewise  Linear,  c)  Saturation,  d) 
Sigmoid  Network,  and  e)  Wavelet  Network,  [25-27]. 

EXPERIMENTAL  SETUP 

The  experiments  were  carried  out  on  a  healthy  male  subject.  The 
objective  was  to  extract  dynamical  models  describing  the 
relationship  between  sEMG-Force  signals.  The  motor  point  of  the 
subject  was  located  using  a  Muscle  Stimulator,  manufactured  by 
Rich-Mar  Corporation  (model  number  HV  1100).  The  motor 
location  of  the  ring  finger  was  identified  and  chosen  for  the 
experiments.  The  EMG  detection  system  used  was  a  Delsys, 
Bagnoli- 1 6  channel  EMG  (DS- 1 60,  S/N- 1 1 1 6),  which  has  internal 
amplification  and  noise  reduction  technology  built  into  it.  The 
sensors  used  for  measuring  the  surface  EMG  action  potentials  are 
two  pronged  DE  2.1  differential  surface  electrodes.  The  material 
for  the  contacts  of  the  electrode  is  99.9%  pure  silver,  while  the 
contacts  are  10mm  long,  1mm  in  diameter  and  spaced  10mm 
apart. 

The  subjects’  skin  was  prepared  before  the  sensors  were  placed 
over  the  motor  point  location  of  the  forearm  musculature  being 
measured.  Hair  around  the  area  was  shaved  and  an  alcohol  swab 
was  used  to  clean  the  skin.  The  Delsys  Electrode  Interface  SC- 
F02-3  was  used  which  promotes  the  quality  of  the  electrical 
connection  between  the  electrode  and  the  skin,  minimizing 
motion  artifacts  and  line  interference.  The  electrodes  were  placed 
along  the  muscle  fibers  (Flexor  Digitorum  Superficialis)  for 
recording  surface  EMG.  The  force  was  recorded  using  a  force 
sensitive  resistor  (FSR)  mounted  on  a  stress  ball.  The  data 
acquisition  for  the  force  and  the  sEMG  signals  was  done  at  2000 
samples  per  second  and  the  NI  DAQ  6024  E  Series  boards  were 
used.  Experiments  were  conducted  for  three  fingers,  the  index, 
middle  and  the  ring  finger.  The  motions  simulated  were  a)  pad  to 
pad  grip  and  b)  a  power  grip,  while  measuring  the  force 
developed  by  individual  fingers.  For  the  ring  finger,  the  motion  to 
generate  the  force  signal  was  randomized  (the  levels  of  force 
amplitude  and  frequency  generated  by  the  subject  varied 
significantly  over  one  experiment  and  for  all  six  experiments  that 
were  carried  out).  This  was  done  in  order  to  have  poor  correlation 
between  the  sEMG-Force  signals  analyzed  for  various  time 
windows.  For  the  index  and  the  ring  finger  four  experiments  were 
conducted  for  each.  The  motion  to  generate  the  force  signal  was 
made  cyclic  in  order  to  have  high  correlation  between  the  sEMG- 
Force  signals.  Cyclic  motion  was  considered,  in  this  case,  to  study 
the  effects  of  fatigue  as  time  progressed  on  system  identification 
and  the  Hammerstein-Wiener  modeling.  Data,  for  the  ring  finger 
(random  motion),  was  collected  for  30,  45  and  60  seconds.  Data 


for  the  middle  and  index  fingers  was  collected  for  30  and  45 
seconds.  The  subject  performed  a  cyclic  and  random  series  of 
flexions  and  contractions  with  the  ring  finger  at  constant  and 
random  speeds.  The  randomness  -  as  depicted  in  Figure  5a)  -  was 
necessary  in  order  to  ensure  that  the  system  is  excited  at  all 
modes.  Figure  5a)  shows  the  output  of  the  random  motion  raw 
sEMG  signal  of  the  ring  finger,  5b)  Chebyshev  filtered  random 
motion  sEMG-Force  signals  (ring  finger),  5c)  Half-Gaussian 
filtered  sEMG,  and  also  the  data.  5d)  cyclic  sEMG-Force  signals 
of  the  middle  finger  and  5e)  cyclic  sEMG-Force  signals  of  the 
index  finger.  The  x-axis  in  the  figures  is  the  number  of  samples 
and  the  y-axis  is  the  amplitude  of  the  sEMG  and  the  force  signal 
in  milli- volts.  During  the  experiment  all  fingers  were  flexed  in 
order  to  generate  the  force  and  sEMG  signal.  A  result  of  this  was 
substantial  cross  talk  in  the  measurement.  This  is  closer  to  a  real 
life  situation.  The  fit  values  computed  for  the  models  uses  the 
relation  in  equation,  (8); 

fit  =  100  *  (1  —  norm(y  -  y)  /  norm(y  -  mean(y)))  (8) 

RESULTS  AND  DISCUSSION 


(e) 


FIGURE  5:  A)  RANDOM  RAW  EMG  RING  FINGER,  B)  CHEBYSHEV 
FILTERED  EMG  AND  FORCE  (RING  FINGER,  RANDOM),  C)  HALF- 
GAUSSIAN  FILTERED  EMG  (RING),  D)  CYCLIC  EMG-FORCE 
MIDDLE  FINGER,  E)  CYCLIC  EMG-FORCE  INDEX  FINGER 

The  force  signal  was  filtered  using  a  low  pass  Butterworth  filter 
using  a  cutoff  frequency  of  3Hz.  The  sEMG  signal  was  also 
filtered  using  a  Chebyshev  Type  II  high  pass  filter  with  the 
following  filter  parameters,  FstOp=60Hz,  Fpass=70,  AstOp=80, 
Apass=l,  match=stopband,  to  remove  any  line  frequencies  and 
followed  by  a  low  pass  Chebyshev  Type  II  high  pass  filter  with 
the  following  filter  parameters,  Fstop=600Hz,  Fpass=650,  Astop=80, 
Apass=l,  match=stopband.  This  was  done  since  EMG  signals  have 
an  effective  frequency  content  of  not  more  than  500Hz,  according 
to  the  ISEK  standards,  [3]. 
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Cyclic  and  Random  Data  Analysis:  As  mentioned  earlier, 
experiments  were  carried  out  a)  in  a  cyclic  motion,  to  have  a 
cyclic  change  in  the  sEMG  and  force  signal  so  that  the  effects  of 
fatigue  could  be  characterized  and  b)  in  a  random  manner  (with 
fatigue  induced  by  repetition)  to  investigate  how  the 
Hammerstein- Wiener  model  parameters  change  with  time.  For  the 
cyclic  case,  experiments  were  conducted  on  the  index  and  the 
middle  finger,  while  the  ring  finger  was  used  for  the  random  case. 
In  all,  for  the  cyclic  case  four  experiments  each  were  conducted 
on  the  index  and  middle  finger  and  the  data  analyzed.  The  sEMG 
data  was  filtered  using  a  Chebyshev  filter  and  a  Half-Gaussian 
filter.  Hammerstein-Wiener  models  were  tested  by  varying  nb  and 
^values  for  both  the  cases.  The  data  set  was  split  into  smaller 
windows;  an  example  of  the  windows  used  for  estimation  and 
validation  of  the  modeling  is  shown  in  Table  1 ,  where  ze  is  the 
time  interval  that  was  used  for  the  model  estimation  and  zv  is  the 
time  interval  used  for  model  validation.  A  small  overlapping  time 
window,  with  the  estimated  model  is  used  for  the  validation 
model. 


TABLE  1:  MODELING  TIME  WINDOWS  CYCLIC  MOTION  (INDEX  AND  MIDDLE 

FINGERS) 


ze  (sec) 

zv  (sec) 

0.5-6. 5 

2. 5-8. 5 

6.5-13.5 

9.5-16.5 

13.5-19.5 

16.5-22.5 

17.0-23.0 

20.0-26.0 

FIGURE  6:  RESULTS  OF  PSO  FOR  SIX  EXPERIMENTS.  AXES  ARE 
ALPHA,  BETA  AND  FIT  VALUES. 

In  all,  the  total  time  intervals  for  which  models  and  fits  were 
computed  for  the  index  finger  alone,  were  4  (time  intervals  per 
experiment)  x  4(experiments)  x  49  (models  per  window  by 

varying  and  «y)x2  (filtering  methods)  =  1,568  different 

models.  This  was  also  repeated  for  the  middle  finger  (1,568 
different  models).  For  the  random  motion/force  variation 
scenario;  8  (time  intervals  per  experiment)  x  7(experiments)  x 
49  (models  per  window  by  varying  n ^  and  nj-)x  2  (filtering 

methods)  x  2  (piecewise  linearity  function,  5  and  10)=  10,976 
different  models  were  analyzed.  It  would  not  be  feasible  to 
show  all  the  results  of  all  the  models  here,  but  examples  are 


provided  as  to  how  the  modeling  was  affected  for  the  various 
time  intervals  and  motion/force  variations.  Also,  for  the  random 
scenario,  PSO  was  used  to  investigate  a  relation  between  the 
filter  parameters  a  and  (3  and  also  to  investigate  the  effect  of 
fatigue  on  the  model  outputs.  The  results  of  the  PSO 
optimization  are  summarized  in  Figure  6.  From  the  results,  it 
can  be  concluded  that  a  values  between  1.5  to  2.9e-6  and  (3 
values  between  2.6-3.7e-2 6  are  more  likely  to  give  better  fits 
for  nonlinear  Hammerstein-Wiener  models.  The  fits  are  not 
very  conclusive,  as  the  fit  values  for  experiment  1  are  not  very 
high  (probably  the  search  space  for  a  and  (3  would  need  to  be 
expanded)  as  compared  to  the  other  experiments  for  the  initial 
time  windows,  where  one  would  not  expect  any  fatigue.  But  a 
trend  can  be  observed  that  as  fatigue  sets  in  towards  the  last 
time  window  of  almost  all  the  experiments  the  fit  %  dropped, 
except  for  experiments  5  and  6  (not  shown  here)  where  the  fits 
were  relatively  high  about  67%.  This  was  because  the 
correlation  between  the  signals  for  these  two  windows  was 
much  higher  than  the  ones  for  the  other  experiments. 

This  indicates  that  as  the  muscle  fatigue  increases,  it  directly 
deteriorates  the  input  sEMG  signals  relation  to  force.  This  casts  a 
doubt  on  the  effectiveness  of  the  nonlinear  blocks  in  the 
Hammerstein-Wiener  models  investigated.  From  these  results,  it 
could  be  concluded  that  a  thorough  investigation  into  the 
nonlinear  block  would  be  needed  to  capture  the  dynamics  of 
fatigue  and  to  compensate  for  the  detrimental  effects  it  has  on 
modeling  the  sEMG-force  signals.  Also,  the  nonlinear  block  could 
be  substituted  for  any  of  the  blocks  that  have  been  discussed  in, 
[28].  Figure  7  shows  some  of  the  results  for  the  index  finger 
nonlinear  Hammerstein-Wiener  models  investigated  for  the 
experiments.  The  red  plot  is  of  the  signal  used  for  model 
estimation,  the  green  plot  is  of  the  signal  used  for  validation  of 
the  model  and  the  blue  plot  is  of  the  entire  data  spectrum  for  that 
experiment  for  Figure  7  (a)  &  (b).  Figure  7(c)  shows  the  auto¬ 
correlation  of  the  model  residuals  for  the  output  signal  (force). 
This  figure  also  plots  the  cross-correlation  of  the  various  models 
between  the  input  (sEMG)  and  the  residuals  of  the  output  (force). 
Figure  7  d)  shows  the  results  of  the  measured  and  the  simulated 
model  output  signal,  for  the  particular  time  window.  The  plots 
shown  here  are  for  the  best  fit  scenarios.  For  example,  nlhwl  with 
the  following  model  parameters;  =  2;  =  3  and  =  1 ,  with 

a  piecewise  linearity  function  value  of  5,  gives  a  fit  of  80.5 1%  for 
the  time  window  0.5-8.0sec  window,  which  was  split  into  0. 5-6.0 
and  2.5-8.0for  model  estimation  and  validation.  The  model 
parameters  were  varied  i.e.  various  n ^  (from  2-9),  and  nj-  (from 

3-10)  were  investigated.  7(e),  (f)  and  (g)  are  the  input 
nonlinearities,  the  linear  block  and  the  output  nonlinearity  blocks 
of  the  Hammerstein-Wiener  models.  The  plot  also  shows  the  case 
where  the  simulated  model  failed  for  a  set  of  parameters  (the  line 
in  the  input  non-linearity  which  goes  across  the  others).  In  the 
linear  block  plot  there  are  a  few  lines  which  have  a  very  large 
deviation  from  the  others;  these  too  are  the  cases  where  the 
modeling  resulted  in  poor  fit  values. 
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FIGURE  7:  A)  EMG  INDEX  FINGER  DATA  SPECTRUM,  B) 
FREQUENCY  CONTENT  OF  SIGNALS.  C)  MODEL  RESIDUALS  AND 
AUTO-CORRELATION  AND  CORSS-CORRELATION  OF  SIGNALS,  D) 
MODEL  OUTPUTS,  E)  INPUT  NONLINEARITY,  F)  LINEAR  BLOCK,  G) 
OUTPUT  NONLINEARITY,  H)  CONTOUR  PLOT  OF  MODEL  FITS 


7(h)  represents  a  contour  plot  of  the  fit  values  for  the  various 
models.  The  description  here,  is  applicable  to  Figure  8  also,  only 
the  results  shown  are  for  the  Middle  finger. 


FIGURE  8:  A)  EMG  MIDDLE  FINGER  DATA  SPECTRUM,  B) 
FREQUENCY  CONTENT  OF  SIGNALS.  C)  MODEL  RESIDUALS  AND 
AUTO-CORRELATION  AND  CORSS-CORRELATION  OF  SIGNALS,  D) 
MODEL  OUTPUTS,  E)  INPUT  NONLINEARITY,  F)  LINEAR  BLOCK,  G) 
OUTPUT  NONLINEARITY,  H)  CONTOUR  PLOT  OF  MODEL  FITS 


Figure  9  shows  a  distribution  of  the  fit  values  for  the  various 
model  parameters  for  a  single  time  window. 


FIGURE  9:  DISTRIBUTION  OF  MODEL  PARAMETRS  AND  THE  FIT 
VALUES  (ONE  TIME  WINDOW) 


CONCLUSION 

From  the  experiments  and  the  Hammerstein- Wiener  modeling  it 
can  be  concluded  that,  a)  the  Hammerstein-Wiener  models  do 
capture  the  dynamics  of  the  sEMG-Force  data,  very  well,  for  all 
the  three  fingers  tested  (in  the  range  of  65  to  82%  model  fits). 
This  statement  holds  true  for  all  the  initial  time  windows  in  the 
experiments.  But  the  model  fits  deteriorate  (between  17-25%)  as 
fatigue  in  the  muscle  increases.  In  order,  to  capture  and  to  account 
for  the  changes  in  the  sEMG-force  signals  due  to  fatigue,  further 
investigation  will  be  needed  into  the  non-linear  block  of  the 
model  structure.  Also  the  size  of  the  windows  chosen  for 
estimation  and  validation  needs  to  have  a  fairly  high  degree  of 
correlation.  For  the  random  motion  case,  where  the  force  was 
varied  randomly,  added  analysis  of  the  frequency  and  amplitude 
of  the  two  signals  was  needed.  The  selection  of  the  size  of  the 
window  could  be  automated  using  correlation  coefficients,  b)  The 
Half-Gaussian  filtering  of  the  sEMG  signal,  used  for  modeling, 
yielded  slightly  better  results  than  the  Chebyshev  filtering  of 
sEMG  (about  7-12%  more),  c)  A  change  in  the  piecewise  linearity 
function  in  the  Hammerstein-Wiener  modeling  did  also  have  a 
significant  effect  on  the  model  fit  value.  Though  a  clear  trend 
could  not  be  seen  from  the  results,  an  adaptive  optimization 
scheme  if  applied  could  help  in  arriving  to  more  concrete 
conclusions  about  the  effect  of  this  parameter  on  the  model  fits,  d) 
Although  the  PSO  did  not  offer  concrete  results  about  the  effect 
of  fatigue  on  the  filter  parameters  a  and  /?,  it  did  throw  some  light 
on  the  direction  in  which  the  search  space  for  these  parameters 
needs  to  be  further  investigated,  which  could  assist  in  modeling 
the  sEMG-force  signals,  especially  with  muscle  fatigue. 
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1.  INTRODUCTION 

The  human  hand  is  considered  to  be  an  agent  of  the  human  brain  and  is  the  most 
intriguing  and  versatile  appendages  of  the  human  body.  Over  the  last  several  years, 
attempts  have  been  made  to  build  a  prosthetic  hand  to  replace  the  human  hand  that 
fully  simulates  the  various  natural /human-like  operations  of  moving,  grasping,  lifting, 
twisting  and  so  on.  Replicating  the  human  hand  in  all  its  various  functions  is  still  a 
challenging  task  due  to  its  extreme  complexity.  It  has  27  bones,  controlled  by  about 
38  muscles  to  provide  the  hand  with  22  degrees  of  freedom  (DOF)  and  incorporates 
about  17,000  tactile  units  of  4  different  units  [99,  207].  Parallels  between  dextrous  robot 
and  human  hands  were  explored  by  studying  sensor  motor  integration  in  the  design  and 
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control  of  these  robots  by  bringing  together  experimental  psychologists,  kinesiologists, 
computer  scientists,  and  electrical  and  mechanical  engineers. 

1.1.  Relevance  to  the  Military 

" In  the  United  States,  there  are  approximately  1.9  million  people  living  with  limb  loss" 
due  to  combat  operations  (such  as  conflicts,  wars,  etc.)  and  non-combat  operations  such 
as  accidents,  or  birth  defects,  according  to  the  Amputee  Coalition  of  America  (AC A)  [3]. 
Further,  during  the  recent  wars  in  Afghanistan  and  Iraq,  "at  least  251,102  were  killed  and 
532,715  seriously  wounded"  [192]. 

The  following  documents  reveal  the  intense  interest  by  military  in  the  area  of  smart 
prosthetics. 

1.  First,  according  to  [131],  recognizing  that  "arm  amputees  rely  on  old  devices"  and 
that  the  existing  technology  for  arm  and  hand  amputees  hasn't  changed  signifi¬ 
cantly  in  the  past  six  decades,  the  Defense  Department  is  embarking  on  a  research 
program  to  "fund  prosthetics  research"  according  to  [132]  to  revolutionalize  upper- 
body  prosthetics  and  to  develop  artificial  arms  that  will  "feel,  look  and  perform" 
like  a  real  arm  guided  by  the  central  nervous  system. 

2.  According  to  [55,  56],  Bio-Revolution  is  one  of  the  eight  strategic  research  thrusts 
that  DARPA  is  emphasizing  in  response  to  emerging  trends  and  national  security. 
In  particular,  the  Human  Assisted  Neural  Devices  program  under  Bio-Revolution 
will  have  "immediate  benefit  to  injured  veterans,  who  would  be  able  to  control 
prosthetics. . .  ."  A  related  area  of  interest  in  Bio-Revolution  is  Cell  and  Tissue 
Engineering. 

3.  Next,  according  to  the  Defense  Science  Office  (DFO)  of  DARPA  [57],  emerging  tech¬ 

nologies  for  combat  casualties  care  with  dual-usage  for  both  military  and  civilian 
medical  care,  focus  on  programs  in  Revolutionizing  Prosthetics,  Human  Assisted 
Neural  Devices,  Biologically  Inspiredd^plti-functional  Dynamic  Robotics,  and  so 
on.  In  particular,  according  to  [58],  one  of  the  most  devastating  battlefield 

injuries  is  loss  of  a  limb ...  at  DARPA,  the  vision  of  a  future  is  to . . .  regain  full  use 
of  that  limb  again." 

In  research  and  development  of  a  prosthetic  hand  using  non-invasive  techniques,  the 
basic  problems  are 

1.  acquising  and  classifying  electromyographic  (EMG)  signals  to  correspond  to  vari¬ 
ous  human-like  operations  or  features, 

2.  converting  EMG  feature  signals  to  control  signals  to  input  to  actuators  to  operate 
the  prosthetic  hand,  and 

3.  designing  and  building  a  prosthetic  hand  with  necessary  sensors  to  operate  accord¬ 
ing  to  the  control  signal  that  corresponds  to  the  EMG  signal  intended  for  the 
particular  identified  hand  operation. 

This  overview  was  first  prepared  in  2006  [138]  when  the  first  author  and  his  colleagues 
at  Idaho  State  University  submitted  a  proposal  on  Smart  Prosthetic  Hand  Technology — 
Phase  I  of  III  to  the  Telemedicine  Advanced  Technology  Research  Center  (TATRC),  which 
is  the  corporate  or  central  laboratory  for  advanced  technology  research  of  the  U.S.  Army 
Medical  Research  and  Material  Command  (USAMRMC)  of  U.S.  Department  of  Defense. 
Since  the  proposal  was  funded  by  the  TATRC  in  July  2007,  the  overview  has  been  contin¬ 
uously  updated  and  was  then  completed  in  the  present  form.  In  this  chapter,  a  chrono¬ 
logical  overview  of  the  applications  of  control  theory  to  prosthetic  hand  is  presented. 
The  overview  focuses  on  hard  computing  or  control  techniques,  such  as  multi-variable 
feedback,  optimal,  nonlinear,  adaptive  and  robust,  and  soft  computing  or  control  tech¬ 
niques,  such  as  artificial  intelligence,  neural  networks,  fuzzy  logic,  genetic  algorithms, 
and  on  the  fusion  of  hard  and  soft  control  techniques.  This  overview,  focusing  on  recent 
developments,  is  intended  to  supplement  the  already  existing  excellent  survey  articles 
[50, 112,  207],  and  is  not  intended  to  be  an  exhaustive  survey  on  this  topic.  Any  omissions 
of  other  works  is  purely  unintentional. 
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2.  CONTROL  STRATEGIES 
2.1.  Prosthetic  Hands 

Over  the  last  several  years  artificial  hands  have  been  developed  by  various  researchers  in 
the  field  and  some  of  the  prosthetic  devices  developed  are  given  below  (in  chronological 
order)  [207,  208]: 

1.  Russian  arm  -  [108,  128,  177] 

2.  Waseda  hand  -  [103] 

3.  Boston  arma  -  [125] 

4.  UNB  hand  (University  of  New  Brunswick)  -  [88,  120,  162] 

5.  Hanafusa  hand  -  [79] 

6.  Crossley  hand  -  [51] 

7.  Okada  hand  -  [146] 

8.  Utah/MIT  hand  (University  of  Utah/Massachusetts  Institute  of  Technology)  -  [95, 
96,  94] 

9.  JPL/Stanford  hand  (Jet  Propulsion  Laboratory/Stanford  University)  -  [167,  195] 

10.  Minnesota  hand  -  [119] 

11.  Manus  hand  -  [114,  157] 

12.  Kobayashi  hand  -  [107] 

13.  Rovetta  hand  -  [165] 

14.  UT/RAL  hand  -  [105] 

15.  Dextrous  gripper  -  [193] 

16.  Belgrade/USC  hand  (University  of  Belgrade /University  of  Southern  California)  - 
[18] 

17.  Southampton  hand  (University  of  Southampton,  Southampton,  UK)  -  [111] 

18.  MARCUS  hand  (Manipulation  And  Reaction  Control  under  User  Supervision)  - 
[113] 

19.  Kobe  hand  (Kobe  University,  Japan)  -  [148] 

20.  Robonaut  hand  (NASA  Johnson  Space  Center)  -  [124] 

21.  NTU  hand  (National  Taiwan  University)  -  [86] 

22.  Hokkaido  hand  -  [145] 

23.  DLR  hand  (Deutschen  Zentrums  fur  Luft-  und  Raumfahrt-German  Aerospace 
Center)  -  [26,  121] 

24.  TUAT/ Karlsruhe  hand  (Tokyo  University  of  Agriculture  and  Technol¬ 
ogy/University  of  Karlsruhe)  -  [74] 

25.  BUAA  hand  (Beijing  University  of  Aeronautics  and  Astronautics)  -  [209] 

26.  TBM  hand  (Toronto /Bloorview  MacMillan)  -  [59] 

27.  ULRG  System  (University  of  Louisiana  Robotic  Gripper)  -  [109] 

28.  Oxford  hand  -  [114] 

29.  IOWA  hand  (University  of  Iowa)  -  [205] 

30.  MA-I  hand  -  [182] 

31.  RCH  -1  (ROBO  CASA  hand  lb)  -  [159] 

32.  UB  hand  (University  of  Bologna)  -  [123] 

33.  Ottobock  SUVA  hand  -  (www.ottobock.co.uk) 

34.  Northwestern  University  system  -  [72] 

35.  SKKU  Hand  II  (Sungkyunkwan  University,  Korea)  -  [44] 

36.  Applied  Physics  Laboratory  (APL)  at  Johns  Hopkins  University  (JHU)  -  [7,  8,  58] 
Some  of  the  commercial  web  sites  for  prosthetic  devices  are: 

1.  Sensor  Hand™  Speed  from  Otto  Bock  (www.ottobock.co.uk), 

2.  VASI  (Variety  Ability  Systems  Inc.),  a  company  of  the 
Otto  Boch  Group  (http://www.vasi.on.ca/index.html). 


flThe  "Boston  Arm,"  project  involved  the  Harvard  Medical  School,  Massachusetts  General  Hospital,  the  Liberty 
Mutual  Research  and  Rehabilitation  Centers,  and  M.I.T. 

&The  Italy-Japan  joint  laboratory  for  Research  on  Humanoid  and  Personal  Robotics. 
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3.  Utah  Arm  from  Motion  Control  (www.utaharm.com), 

4.  The  i-LIMB  Hand  from  Touch  Bionics  (www.touchbionics.com),  and  so  on. 

A  very  useful  comparison  table  between  several  hands  listed  above  and  the  human 
hand,  adapted  from  [207,  208],  is  updated  and  shown  in  Table  1. 

However,  about  35%  of  the  amputees  do  not  use  their  prosthetic  hand  regularly  accord¬ 
ing  to  [14]  due  to  various  reasons  such  as  poor  functionality  of  the  presently  available 
prosthetic  hands  and  psychological  problems.  To  overcome  this  problem,  one  has  to 
design  and  develop  an  artificial  hand  that  "mimics  the  human  hand  as  closely  as  possi¬ 
ble"  both  in  functionality  and  appearance. 

There  are  a  number  of  surveys,  and/or  state-of-the  art  articles  that  appeared  over  the 
years  on  the  subject  of  myoelectric  prosthetic  hand  including  the  work  in  USSR  (Russian) 
[177]  and  other  places  [10,  19,  21,  43,  50,  77,  93,  112,  137,  147,  173,  174,  178,  207]. 

2.2.  Chronological  Overview 

Up  to  1970: 

Electromyographic  (EMG)  signal  is  a  simple  and  easily  obtainable  source  of  informa¬ 
tion  about  the  various  movements  to  be  used  for  artificial /prosthetic  hands.  The  EMG 
extraction  using  surface  electrodes  is  a  very  attractive  method  from  the  point  of  view 
of  the  user  compared  to  implants  requiring  surgery.  Research  activity  in  the  field  of 


Table  1.  Comparison  of  human  hand  with  artificial  hands:  robotic  and  prosthetic  hands.  Force  indicates  power 
grasp.  Speed  indicates  the  time  required  for  a  full  closing  and  opening,  E,  stands  for  external ,  I,  stands  for 
internal. 


Size 

(Norm) 

#  of 
fingers 

#  of 
DOF 

#  of 

sensors 

#  of 

actuators 

Weight 

(gins) 

Force 

(N) 

Speed 

(sec) 

Controls 

Human  hand 

Russian  arm 

Waseda  hand 
f T  ™ TB  hand 
lz^_Jafusa  hand 
Crossley  hand 

1.0 

5 

5 

22 

^17,000 

3 

38(I+E) 

1 

~400 

>300 

^147 

0.25 

E 

Utah/MIT  hand 

^2.0 

4 

16 

16 

32(E) 

— 

31.8 

— 

E 

JPL/ Stanford  hand 

«1.2 

3 

9 

— 

12(E) 

1100 

45 

— 

E 

Minnesota  hand 


lyashi  hand 
itta  hand 


UT/RAL  hand 

Dextrous  gripper 
Belgrade/USC  hand 

«1.1 

4 

4 

23+4 

4(E) 

E 

Southampton  hand 

«1.0 

5 

6 

— 

6(E) 

400 

38 

^5 

E 

MARCUS  hand 

«1.1 

3 

2 

3 

2(1) 

— 

— 

— 

I 

Kobe  hand 

Robonaut  hand 

wl.5 

5 

12+2 

43+ 

14(E) 

E 

NTU  hand 

«1 

5 

17 

35 

17(E) 

1570 

— 

— 

E 

Hokkaido  hand 

«1 

5 

17 

35 

17(E) 

1570 

— 

— 

E 

DLR  hand  II 

>1 

5 

7 

— 

7(E) 

125 

— 

— 

E 

TUAT /Karlsruhe  hand 

«1 

5 

17 

17(E) 

^120 

12 

0.1 

E 

BUAA  hand 
r7"1  hand 
vrz_Jrd  hand 

IOWA  hand 

MA-I  hand 

Robo  Casa  hand-1 

«1 

4 

5 

16 

24 

2 

6(E+I) 

350 

~40 

0.25 

E 

Ottobock  SUVA 

1 

3 

1 

2 

1(E) 

600 

— 

— 

I 

UB  hand 

Hokkaido  hand 

>1 

5 

7 

7(E) 

125 

E 

Northwestern 

SKKU  Hand  II 

APL-JHU  System 

1.1 

4 

4 

— 

3 

900 
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prosthetic  limbs  was  initiated  by  the  US  National  Academy  of  Sciences  in  response  to 
the  needs  of  a  large  number  of  casualties  in  World  War  II  [135].  Researchers  reporting  in 
Refs.  [16,  203]  first  proposed  the  concept  of  EMG  signals  for  the  control  of  a  prosthetic 
hand  for  amputees.  A  proportional  (open-loop)  control  systems  in  which  the  amplitude 
of  the  hand  motor  voltage  and  hence  its  speed  and  force  measured  from  strain  gauges 
varies  in  direct  proportion  (linearly)  to  the  amplitude  of  the  EMG  signal  generated  by 
the  prosthetic  hand,  was  first  reported  in  Refs.  [24,  25].  In  addition,  the  system  added 
force  and  velocity  feedback  controls  so  the  users  could  feel  more  natural  when  using  this 
device.  An  adaptive  control  scheme  was  developed  by  [187]  for  the  Southampton  Hand. 
1971-79: 

The  work  reported  in  Ref.  [162]  studied  the  effect  of  sensory  feedback  based  on  semi¬ 
conductor  strain  gauges  on  either  edge  of  thumb  of  the  prosthetic  hand  to  adjust  the 
stimulus  magnitude  to  target  value  and  avoid  dropping  or  crushing  objects  for  control 
of  a  prosthesis  and  found  this  acceptable  for  patients.  When  the  strain  gauges  received 
the  stimulus,  the  system  amplified  and  transferred  the  signals  to  comparator,  and  then 
the  comparator  modified  the  range  of  amplitude  of  stimulus  to  the  level  that  the  users 
needed.  However,  the  device  with  feedback  is  two  or  three  times  larger  than  the  normal 
hand.  A  hierarchical  method  consisting  of  analytical  control  theory,  such  as  performance- 
adaptive  self-organizing  control  algorithm  and  artificial  intelligence  using  fuzzy  automa¬ 
ton,  was  presented  in  Ref.  [169]  to  drive  a  prosthetic  hand. 

1980-89: 

In  providing  a  historical  perspective,  the  contribution  in  Ref.  [43]  presented  the  status 
of  the  closed-loop  (feedback)  control  principles  for  the  application  of  prosthetic  devices. 
Three  concepts  relating  to  supplemental  sensory  feedback,  artificial  reflexes,  and  feed¬ 
back  through  control  interfaces  were  discussed,  and  it  was  concluded  that  "we  have 
not  moved  very  far  in  the  last  65  years  in  the  clinical  application  of  these  concepts." 
A  statistical  analysis  involving  the  study  of  zero-crossings,  second  to  fifth  moments  and 
correlation  functions  and  pattern  classification  of  EMG  signals  was  given  in  Ref.  [168]. 
A  probabilistic  model  of  the  EMG  pattern  was  formulated  in  the  feature  space  of  integral 
absolute  value  (IAV)  to  provide  the  relation  between  a  command,  represented  by  motion 
and  speed  variables  and  the  location  and  shape  of  the  pattern  for  real-time  control  of  a 
prosthetic  arm  as  given  in  Ref.  [118].  Using  kinematic  relationships  for  dynamic  model 
of  fingers,  multi-variable  feedback  control  strategies  using  pole  assignment  in  frequency 
domain  were  employed  by  [195]  to  guarantee  local  stability  for  controlling  one  finger  of 
the  JPL/ Stanford  hand.  The  work  in  Ref.  [195]  produced  the  dynamic  models  of  three 
fingers  (thumb,  index,  and  middle)  and  three  joints  first,  and  then  used  Laplace  trans¬ 
form  to  work  in  frequency  domain.  To  get  a  guaranteed  stability  of  control  system,  the 
roots/ poles  had  to  be  located  in  the  left  half  plane.  Hence,  they  could  get  a  desired  steady 
movement  of  fingers  by  controlling  the  positions  of  the  roots.  The  works  reported  by  the 
groups  in  Refs.  [49,  136]  were  the  first  to  investigate  various  aspects  such  as  kinemat¬ 
ics,  prehensility,  dynamics,  and  control  of  multi-fingered  hands  manipulating  objects  of 
arbitrary  shape  in  three  dimensions. 

1990-99: 

Design,  implementation,  and  experimental  verification  of  an  improved  cybernetic 
elbow  prosthesis  was  presented  in  Refs.  [1,  2]  that  mimics  the  natural  limb  to  both 
internal  (voluntary)  inputs  from  the  amputee  and  external  inputs  from  the  environment. 
The  work  in  Ref.  [181]  considered  a  dextrous  hand  employing  a  systematic  approach 
to  achieve  the  object  stiffness  control  by  actuator  position  control,  tendon  tension  con¬ 
trol,  joint  torque  control,  joint  stiffness  control,  and  Cartesian  fingertip  stiffness  control. 
The  work  in  Ref.  [174]  conducted  a  survey  of  33  patients  wearing  the  proportional  myo¬ 
electric  hand  grouped  into  3  categories  based  on  previous  experience  with  a  terminal 
device:  digital  (on-off)  myoelectric  hand,  body-powered  terminal  device,  and  no  termi¬ 
nal  device.  The  survey  results  showed  that  the  group  of  patients  having  experience  with 
the  digital  hand  "were  most  impressed  with  proportionally  controlled  hand,"  because  it 
has  the  advantages:  comfortable,  cosmetic  acceptance,  more  natural,  superior  pinch  force 
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(11-25  lb)  compared  to  voluntary  opening  (7-8  lb),  a  greater  range  of  function  but  less 
energy,  sensory  feedback,  force  feedback,  and  short  below-elbow. 

The  research  work  in  Ref.  [81]  developed  three  tests  for  evaluation  of  input-output 
properties  of  patient  control  of  neuroprosthetic  hand  grasp,  which  can  compensate  or 
enrich  the  function  of  a  damaged  peripheral  nervous  system.  The  first  test  was  for  static 
input-output  properties  of  the  hand  grasp,  the  second  one  was  for  control  of  hand  grasp 
outputs  while  tracking  step  and  ramp  functions,  and  the  final  test  was  to  obtain  the  input- 
output  frequency  response  of  the  hand  grasp  system  dynamics  to  estimate  the  transfer 
function  using  spectral  analysis.  Each  test  used  visual  feedback  when  the  users  controlled 
the  grasp  force  and  grasp  position  tracking  of  the  hand.  It  was  shown  in  Ref.  [89]  that 
the  myoelectric  signal  is  not  random  during  the  initial  phase  of  muscle  contraction  thus 
providing  a  means  of  classifying  patterns  from  different  contraction  types.  The  means  is 
to  establish  the  60  records  of  an  isometric  contraction  of  the  subjects  and  then  produce 
some  anisometric  contraction  types,  like  flexion  and  extension.  This  information  was  use¬ 
ful  in  designing  a  new  multi-function  myoelectric  control  system  using  artificial  neural 
networks  (ANN)  for  classifying  myoelectric  patterns.  Additionally,  the  hidden  layer  size, 
segment  length,  and  EMG  electrode  positions  were  studied.  See  related  works  in  Refs. 
[68,  69,  75,  76]  on  multi-functional  myoelectric  control  systems  using  pattern  recognition 
methods  for  myoelectric  signal  extraction  and  classification.  The  control  philosophy  of  a 
multi-fingered  robotic  hands  for  possible  adaptation  and  use  in  prosthetics  and  rehabili¬ 
tation  was  discussed  in  Refs.  [90-92]  with  respect  to  the  Belgrade /USC  robot  hand  [18], 
called  PRESHAPE  (Programmable  Robotic  Experimental  System  for  Hands  and  Prosthet¬ 
ics  Evaluation),  which  can  estimate  a  system  that  translates  task  commands  to  motor 
commands  using  pressure  sensors,  force  sensors  and  pressure  feedback,  which  is  very 
useful  to  detect  small  contact  forces. 

Using  the  dynamic  model  of  the  nonlinear  neuromuscular  (motor  servo)  control  sys¬ 
tem  of  human  finger  muscles  including  mechanical  properties  (such  as  viscoelasticity) 
of  the  muscle  and  stretch  reflex,  a  surface-based  myoelectrically-controlled  biomimetic 
prosthetic  hand  (called  the  Kobe  hand)  with  three  fingers  (thumb,  index  and  middle 
fingers)  was  developed  at  Kobe  University,  Japan  [148]  with  a  system  consisting  of  an 
EMG  signal  processing  unit,  the  dynamic  model,  a  positional  control  unit,  and  the  pros¬ 
thetic  device.  A  survey  of  four  important  properties — dexterity,  equilibrium,  stability, 
and  dynamic  behavior  relating  to  autonomous  multi-fingered  robotic  hands — was  pre¬ 
sented  [178].  An  interesting  aspect  of  this  literature  survey  is  a  series  of  tables  relating  to 
existing  multi-fingered  robotic  hands,  force  closure,  dexterity  in  kinematically  redundant 
robotic  hands,  equilibrium,  in  robotic  grasp,  and  stability.  As  reported  in  Ref.  [15],  an 
intelligent  prosthesis  control  system,  developed  by  Animated  Prosthetics,  consists  of  two 
parts:  the  animation  control  system  (ACS)  residing  in  prosthesis  and  a  remote  prosthe¬ 
sis  configuration  unit  (PCU)  capable  of  on/off  to  variable  speed/grip.  Dynamic  control 
of  two  arms  to  manipulate  cooperatively  an  object  with  rolling  contacts  was  addressed 
[170]  using  a  nonlinear  feedback  control  methodology  that  decouples  and  linearizes  the 
system. 

A  sensory  control  system  based  on  a  force  sensing  resistor  (FSR)  was  developed  [191] 
at  The  National  Institute  for  Accidents  at  Work  (IN AIL),  Bologna,  Italy,  to  control  the 
strength  of  the  grip  on  objects  for  a  commercial  prosthetic  hand  having  two  main  func¬ 
tions:  the  automatic  search  for  contact  with  the  object  and  the  detection  of  the  object 
possibly  slipping  the  grip  by  involuntary  feedback  (force  sensors  and  slipping  sensors). 
Further,  automatic  tuning  of  control  parameters  of  the  prostheses  was  investigated  [23] 
using  fuzzy  logic  expert  systems  resulting  in  a  software  package  called  the  microproces¬ 
sor  controlled  arm  auto  tuning.  The  automatic  tuning  software  works  as  follows: 

•  the  client  connects  the  prosthesis  hardware 

•  the  program  needs  both  sensor  signals  as  client  input  the  program  combines  the 
above  qualitative  and  quantitative  information  stored  in  the  fuzzy  logic  database  to 
calculate  the  prosthesis  parameter  values 

•  the  program  enables  the  new  parameter  values  to  be  down-loaded  into  the  prosthesis 
control  system  memory. 
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Dynamic  modeling  of  a  robotic  hand  was  proposed  [172]  using  a  hybrid  approach  with 
a  discrete  event  aspect  of  grasping  and  continuous-time  part  with  a  variable  structure 
impedance  control  algorithm.  A  novel  on-line  learning  method  was  reported  in  Ref.  [145] 
for  prosthetic  hand  control  based  on  EMG  measurements  with  a  system  consisting  of 
three  units: 

•  an  analysis  unit  for  generating  feature  vectors  containing  useful  information  for 
discriminating  motions  from  EMG  signals 

•  an  adaptation  unit  for  adapting  to  the  amputee's  individual  variation  and  for  dis¬ 
criminating  motions  from  the  feature  vector  and  at  the  same  time  generating  the 
necessary  control  commands  to  the  prosthetic  hand 

•  a  trainer  unit  for  directing  the  adaptation  unit  to  learn  in  real  time  based  on  the 
amputee's  teaching  signal  and  the  feature  vector. 

In  Ref.  [191]  researchers  built  a  sensory  control  system  based  on  the  force-sensing  resis¬ 
tor  (FSR)  for  an  upper  limb  prosthesis  and  an  optical  sensor  for  detecting  movement.  The 
prostheses  produced  were  of  the  "all  or  nothing"  (opening  or  closing)  and  proportional 
control  type  (the  relationship  between  force  and  EMG  signal  is  linear).  For  traditional 
control,  it  used  voluntary  (visual)  feedback,  but  the  users  had  to  pay  close  attention.  This 
work  developed  an  involuntary  feedback  control  that  uses  two  kinds  of  sensors,  strength 
and  slipping  sensors.  If  the  prosthesis  hand  is  slipping,  the  control  system  can  automat¬ 
ically  order  the  actuator  of  the  prosthesis  to  increase  the  grip  strength.  On  receiving  the 
EMG  signal,  the  hand  begins  a  closing  action  and  goes  on  closing  until  the  FSRs  pro¬ 
duce  a  signal  that  is  greater  than  or  equal  to  a  "contact  threshold"  value.  Then  it  stops 
because  the  object  has  been  grasped  with  the  required  strength  of  grip.  The  automatic 
grip  mechanism  is  very  useful  in  grasping  delicate  objects. 

The  investigation  in  Ref.  [130]  showed  that  the  proposed  neuro-fuzzy  classifier  known 
as  Abe-Lan  network,  is  able  to  identify  correctly  all  the  EMG  signals  related  to  different 
movements  of  the  human  hand.  A  highly  anthropomorphic  human  hand,  called  Robo- 
naut  Hand  consisting  of  five  fingers  and  fourteen  independent  degrees  of  freedom,  was 
built  at  NASA  Johnson  Space  Center  to  interface  with  extra-vehicular  activity  (EVA)  crew 
interfaces  onboard  the  International  Space  Station  (ISS)  [124]. 

2000-2007: 

In  Ref.  [190],  estimating  muscular  contraction  levels  of  flexors  and  extensors  using  neu¬ 
ral  networks,  a  new  impedance  control  technique  [83]  was  developed  to  control  impedance 
parameters  such  as  the  moment  of  inertia,  joint  stiffness,  and  viscosity  of  a  skeletal  mus¬ 
cle  model  of  a  prosthetic  hand.  An  overview  of  dextrous  manipulation  gave  [147]  an 
interesting  time-line  chart  for  the  development  of  robotic  dextrous  manipulation  during 
the  period  1960s  to  2000.  An  excellent  survey  [19]  summarizing  the  evolution  and  state 
of  the  art  in  the  robotic  hands  focusing  mainly  on  functional  requirements  of  manipu¬ 
lative  dexterity,  grasp  robustness,  and  human  operability.  Also,  the  work  in  Ref.  [126] 
exploited  the  nonholonomic  character  of  a  pair  of  bodies  with  regular  rigid  surfaces 
rolling  onto  each  other,  to  study  the  constructive  controllability  algorithm  for  planning 
rolling  motions  for  dextrous  robot  hands.  A  control  system  architecture  was  proposed 
[133, 134]  with  a  feedforward  loop  based  on  EMG  measurements  consisting  of  a  low-pass 
filter  and  neural  network  to  provide  the  actual  torque  signal  and  a  feedback  loop  based 
on  the  desired  angle  consisting  of  a  proportional-derivative  (PD)  controller  to  provide 
the  desired  torque  signal  and  the  error  signal  between  these  torques  drives  the  prosthetic 
hand  to  achieve  the  desired  angle  while  the  neural  network  learns  based  on  feedback 
error. 

Peckham  et  al.  [155]  studied  finger  extension,  external  control,  overhead  reach,  and 
forearm  pronation.  For  finger  extension,  they  used  two  electrodes:  one  placed  between 
the  2nd  and  3rd  metacarpals  and  the  other  between  the  3rd  and  4th  metacarpals.  They 
could  provide  full  extension  of  the  index,  long,  and  ring  fingers.  For  external  control,  a 
new  form  of  control  was  developed  by  using  retained  voluntary  wrist  extension  to  con¬ 
trol  grasp  opening  and  closing.  Overhead  reach  can  be  provided  by  stimulation  of  the 
triceps  muscle,  so  elbow  position  can  be  controlled  by  voluntary  activation  of  biceps  as 
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an  antagonist.  As  for  forearm  pronation,  the  main  issues  are  an  increased  number  of  stim¬ 
ulus  channels  to  allow  stimulation  of  the  finger  intrinsic  muscles,  triceps,  and  forearm 
pronator,  an  implanted  control  source,  bidirectional  communication  between  sensor  and 
body,  reduced  size,  and  reduction  of  all  external  cables.  The  work  in  Ref.  [207]  presents  a 
review  of  the  traditional  methods  for  control  of  artificial  hands  using  EMG  signals  in  both 
clinical  and  research  areas  and  points  out  future  developments  in  the  control  strategy  of 
the  prosthetics,  in  particular  advocating  neuroprosthesis  with  biocompatible  neural  inter¬ 
face  for  providing  sensory  feedback  to  the  user  leading  to  electroneurographic  (ENG) 
based  control  in  place  of  electromyographic  (EMG)  control. 

Collaboration  between  the  University  of  Southampton  and  University  of  New 
Brunswick  (UNB)  [120]  resulted  in  a  hybrid  control  system  using  a  multilayer  perceptron 
(MLP)  artificial  neural  network  as  a  classifier  of  time-domain  features  set  (zero-crossings, 
mean  absolute  value,  mean  absolute  slope,  and  trace  length)  extracted  from  myoelectric 
signals  (MES)  and  a  digital  signal  processor  (DSP)  controlling  the  grip  pressure  of  the 
prosthetic  hand  without  visual  feedback  (voluntary  feedback).  Design  and  development 
of  an  underactuated  (the  number  of  actuators  less  than  the  degrees-of-freedom)  mecha¬ 
nism  applicable  to  prosthetic  hand  was  presented  in  Ref.  [127]  based  on  dynamic  model 
of  fingers  leading  to  adaptive  grasp  (that  is,  being  able  to  conform  to  the  shape  of  an 
object  held  within  the  hand). 

Although  an  adaptive  control  scheme  was  developed  [187]  for  a  Southampton  Hand, 
further  developments  were  made  in  the  research  first  generation  [22,  112]  producing 
their  IP  (intelligent  prosthesis)  according  to  Ref.  [111].  The  investigation  [116]  provided 
an  evolution  of  microprocessor-based  control  systems  for  prosthetics  classified  as  (based 
on  digital  systems),  second  generation  (with  low  power),  and  third  generation  (based 
on  microprocessors  and  digital  signal  processors).  The  work  in  Ref.  [114]  conducted  a 
comparison  of  Oxford  and  Manus  hand  prostheses  with  respect  to: 

1.  hand  mechanisms 

2.  control  electronics:  EMG  analog  amplifiers,  A/D  converters,  digital  signal 
processors 

3.  sensors:  force,  position  and  slip  sensors  based  on  Hall  effect 

4.  manipulation  or  control  schemes:  Oxford  hand  used  Southampton  Adaptive 
Manipulation  Scheme  consisting  of  a  three-level  hierarchical  scheme  and  Manus 
used  a  two-level  scheme. 

The  scheme  suggested  in  Ref.  [30]  consisted  of  five  modules,  including  an  artificial 
muscoskeletal  system,  position  and  force  sensors,  3D  force  sensors,  low-level  control  loop 
dedicated  to  control  slipping  and  grasping,  and  an  EMG  control  unit.  Further,  the  scheme 
used  two  semiconductor  strain  gauges  as  the  force  sensor  and  the  sensor  in  SS496B  by 
Honeywell  International,  Inc.  as  the  position  sensor,  which  is  the  linear  slider  and  small 
magnets.  Moreover,  the  control  system  receives  three  signals:  activation  (EDG  used  to 
identify  whether  there  is  a  movement),  direction  (SGN  decides  opening  or  closing),  and 
amplitude  of  the  movement  (AMP  controls  the  seed  of  the  movement  in  a  proportional 
means).  As  for  the  control  scheme,  it  uses  a  simple  proportional  open-loop  control. 

A  cylindrical  grasp  of  a  cylindrical  object  and  a  parallel  force /position  control  is  stud¬ 
ied  by  [171]  to  ensure  the  stability.  The  work  in  Ref.  [156]  presented  a  feedback  control 
system  for  hand  prosthesis  with  elbow  control.  Using  a  concept  of  extended  physiological 
proprioception  (EPP)  (i.e.,  using  natural  physiological  sensors),  both  the  work  [156]  and 
the  investigation  in  Ref.  [6]  developed  microprocessor-based  controllers  for  upper  limb 
prostheses.  A  systematic  literature  review  [194]  is  useful  for  prosthetic  hand,  although 
the  survey  was  done  for  lower-limb  prosthesis.  Scherillo  et  al.  [171]  developed  a  proce¬ 
dure  to  obtain  maximum  load  and  contact  force  distribution  for  a  given  grasp  task  and 
a  parallel  force /position  control  to  ensure  stability  of  the  grasp.  The  goal  of  this  control 
scheme  is  to  specify  a  set  of  joint  torque  inputs  so  that  the  desired  grasping  forces  along 
the  constrained  directions,  and  the  desired  position  trajectory  along  the  unconstrained 
directions  are  realized. 
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Several  researchers  showed  [9-11]  that  sensory  feedback  signals  are  obtained  for  multi¬ 
fingered  robot  hands  to  perform  the  function  of  grasping  an  object  and  that  dynamic 
force /torque  closure  can  be  constructed  without  knowing  object  kinematic  parameters 
and  location  of  the  mass  center.  Further,  the  convergence  of  motion  of  the  overall  fingers- 
object  system  was  proved  using  the  concepts  of  "stability  and  asymptotic  stability  on 
a  manifold."  Mechanical  design  and  manipulation  (control)  issues  were  addressed  [157] 
for  a  multi-fingered  dextrous  hand  for  upper  limb  prosthetics  using  the  underactuated 
kinematics  enhancing  the  performance  and  providing  four  grasping  modes  (cylindrical, 
precision,  hook,  and  lateral)  with  just  two  actuators,  one  for  the  thumb  and  one  for  the 
remaining  fingers.  In  particular,  the  hierarchical  control  architecture  consists  of  a  host  (or 
master)  controller  for  EMG  management  and  definition  of  grasp  set  points  (for  position 
and  torque /force)  and  three  local  (or  slave)  controllers  for  low-level  implementation  of 
stiffness  control  of  the  joints.  In  Ref.  [205],  design  and  analysis  was  presented  for  a  multi¬ 
fingered  prosthetic  hand  consisting  of  a  thumb  with  three  joints  and  the  rest  of  the  four 
fingers  having  two  joints  using  Haringx  and  element  stiffness  models,  which  enables 
the  location  of  actuators  far  away  from  the  hand  to  a  belt  around  the  waist  and  further 
enabling  actuation  and  control  with  relatively  high  degrees  of  freedom.  Robotic  hand 
MA-I  was  designed  and  built  [182]  at  the  Institute  of  Industrial  and  Control  Engineering 
(IOC)  at  the  Polytechnic  University  of  Catalonia  (UPC)  with  16  degrees  of  freedom  and 
the  control  system  consisting  of  16  position  control  loops,  independently  controlling  each 
of  the  16  DC  motors.  Visual  hand  motion  capture  is  a  multiple-dimension  and  multiple- 
objective  searching  optimization  problem  and  the  work  reported  in  Ref.  [53]  used  pose 
estimation  and  a  motion-tracking  scheme  with  genetic  algorithms  (GA)  embedded  par¬ 
ticle  filter  (PF)  to  navigate  visual  hand  gesture,  such  as  virtual  environment  and  control 
of  a  robot  arm. 

The  fabrication  of  a  compliant,  under-actuated  prosthetic  hand  (both  palm  and  fin¬ 
gers)  molded  as  a  soft  polymeric  single  part  for  providing  adaptive  grasp  was  reported 
[28,  29].  The  analysis  and  synthesis  are  "so  complex  and  only  experimental  analysis  of 
the  solution  adapted  validate  our  works."  It  was  shown  [154]  that  an  object  with  parallel 
surfaces  in  a  horizontal  plane  could  be  controlled  by  a  pair  of  robotic  fingers  to  achieve 
stable  grasping,  angle  and  position  control  without  the  need  for  the  object  parameters  or 
object  sensors  such  as  tactile,  force,  or  visual  sensors. 

At  Northwestern  University  Prosthetics  Laboratory  (NUPL),  the  researchers  [199,  201] 
developed  multi-function  prosthetic  hand /arm  controller  system  receiving  signals  from 
as  many  as  sixteen  implantable  myoelectric  sensors  (IMES)  and  a  heuristic  fuzzy  logic 
approach  to  EMG  signal  pattern  recognition  [5,  200].  In  particular,  fuzzy  logic  was 
explored  for  discriminating  between  multiple  surface  EMG  control  signals  and  classify¬ 
ing  them  to  user  intention.  The  multi-functional  hand  mechanism  consisted  of  a  three- 
motor  hand  (one  motor  for  driving  the  thumb,  one  motor  for  the  index  finger,  and  the 
third  motor  for  the  middle,  ring  and  little  finger)  and  a  two-motor  wrist  (one  motor 
for  wrist  extension/ flexion  and  the  other  motor  for  wrist  rotation).  Further,  the  research 
[72]  demonstrated  that  in  implementing  the  extended  physiological  proprioception  (EPP) 
control  for  a  powered  prosthesis,  the  backlash  is  determined  by  the  stiffness  of  the  con¬ 
trol  cable  as  well  as  mass  located  at  the  distal  end  of  the  forearm  and  that  reduction  of 
static  friction  and  backlash  in  the  system  could  prevent  the  limit  cycle. 

It  was  demonstrated  [60]  that  by  implanting  electrodes  within  individual  fascicles  of 
peripheral  nerve  stumps,  appropriate,  distally  referred  sensory  feedback  about  joint  posi¬ 
tion  and  grip  force  from  an  artificial  arm  could  be  provided  to  an  amputee  through 
stimulation  of  the  severed  peripheral  nerves,  which  also  provide  appropriate  signals.  It  is 
interesting  to  note  [100]  the  mechanism,  design,  and  control  system  of  a  humanoid-type 
hand  with  human-like  manipulation  capabilities  as  a  part  of  the  development  of  service 
robots  and  the  comparison  (shown  by  [101,  158])  of  natural  and  prosthetic  hands.  In 
Ref.  [211],  the  EMG  motion  pattern  classifier  was  developed  using  a  parametric  autore¬ 
gressive  (AR)  model  and  Levenberg-Marquardt  (LM)-based  neural  networks  to  identify 
three  types  of  motion  of  thumb,  index,  and  middle  fingers  to  control  a  five-fingered 
underactuated  prosthetic  hand. 
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The  work  in  Ref.  [161]  focused  on  the  " optimal"  delay  as  the  maximum  amount  of 
time  from  command  to  hand  movement,  for  a  prosthesis  controller  with  a  delay  of  200  to 
400  ms  as  the  range  acceptable  by  users.  A  bypass  prosthesis,  called  the  Prosthetic  Hand 
for  Able-Bodied  Subjects  (PHABS),  was  developed  to  allow  able-bodied  subjects  to  oper¬ 
ate  a  prosthetic  terminal  device.  The  controller  is  a  commercially  available  Myo-pulse 
control,  which  combines  pulse  width  modulation  (PWM)  and  pulse  period  modulation 
(PPM)  because  it  provides  a  linear  relation  between  motor  speed  and  the  pulse  width 
and  timing  of  a  digital  control  signal.  In  addition,  it  also  used  a  mechanical  low-pass 
filter  to  smooth  the  pulse  train  and  movement.  If  the  EMG  reaches  the  threshold,  the 
motor  will  be  turned  "on";  otherwise,  it  will  be  turned  "off."  Furthermore,  the  experi¬ 
mental  controller  was  created  in  Matlab's  Simulink  and  executed  using  Simulink's  Real 
Time  and  XPC  Target  Toolboxes.  Finally,  this  work  summarized  seven  time-delay  sources, 
including: 

1.  the  time  from  the  intent  of  movement  to  the  development  of  EMG 

2.  the  time  constant  of  the  analog  filters  contained  in  the  EMG  pre-amplifiers 

3.  the  analog  to  digital  sampling  period 

4.  the  time  required  to  collect  the  EMG  signal  for  feature  extraction 

5.  the  time  required  to  perform  the  EMG  signal  for  feature  extraction 

6.  the  time  required  to  execute  the  pattern  recognition  on  the  extracted  features 

7.  the  time  required  to  actuate  the  component 

In  Ref.  [50],  a  review  of  the  traditional  methods  of  control  as  well  as  the  current  state  of 
new  control  techniques  was  provided.  A  newly  developed  intelligent  flexible  hand  sys¬ 
tem  with  3  fingers  and  10  joints,  fitted  with  a  small  harmonic  drive  gear  and  a  high  power 
mini  actuator,  providing  12  DOF  applied  to  a  catching  task  was  developed  [117].  The 
authors  [212]  developed  an  EMG-based  (using  electrodes  and  torque  and  angle  sensors) 
prosthetic  hand  control  system  composed  of  a  human  operator,  a  five-fingered  under¬ 
actuated  prosthetic  hand  system,  the  prosthetic  hand  controller  (with  analog-to-digital 
converters,  a  digital  signal  processing  (DSP)  board,  and  stepper  motors)  and  visual  feed¬ 
back.  In  particular,  the  EMG  signals  undergo  feature  extraction  and  feature  classification 
using  neural  networks  with  parametric  autoregressive  (AR)  model  and  wavelet  trans¬ 
forms.  In  an  under-actuated  system,  there  is  a  lower  number  of  actuators  compared  to 
the  number  of  degrees-of-freedom  of  the  system.  Further  in  Ref.  [210],  a  hierarchical  con¬ 
trol  system  was  proposed  with  a  high-level  supervisory  controller  for  implementing  the 
EMG  signal  acquisition  and  pattern  recognition  and  also  providing  a  set  of  commands 
(for  operations  such  as  close,  open,  position,  etc.)  to  a  low-level  controller.  A  sensor-based 
hybrid  control  strategy  (using  normal  feedback  control  based  on  EMG  signals  from  sen¬ 
sors  and  feedback  to  the  user)  was  presented  [160]  where  a  digital  controller  operating 
from  prosthetic  signals  converts  the  user  grasping  intention  (EMG  signal)  into  an  order 
to  control  the  prosthesis. 

Choi  et  al.  [44]  investigated  developed  a  robot  hand  with  tactile  sensors  (slip  sensor 
and  force  sensor),  called  SKKU  Hand  II,  having  two  functional  units:  a  polyvinylidene 
fluoride  (PVDF)-based  slip  sensor  designed  to  detect  slippage  and  a  thin  flexible  force 
sensor  that  can  read  the  contact  force  of  and  geometrical  information  on  the  object  using 
a  pressure  variable  resistor  ink.  A  biomechatronic  approach  to  the  design  and  control  of 
an  anthropomorphic  artificial  hand  was  studied  [216]  for  closing  the  hand  finger  while 
grasping  an  object  using  a  reference  trajectory  and  using  two  different  versions  (joint 
space  and  slider  space)  of  proportional-derivative  (PD)  control  system.  In  particular  the 
artificial  hand  consists  of  three  under-actuated  fingers  (index,  middle,  and  thumb),  which 
are  actuated  by  three  cable-driven  DC  motors  placed  in  the  lower  part  of  the  arm. 

Farrell  and  Weir  [71]  studied  large  controller  delays  created  by  multi-functional  pros¬ 
thesis  controllers.  A  device  called  PHABS  (Prosthetic  Hand  for  Able  Bodied  Subjects)  was 
utilized  to  test  the  performance  of  20  able-bodied  subjects  on  the  Box  and  Block  Test. 
To  estimate  and  compare  the  performance  of  prosthetic  hands,  a  functionality  index  has 
been  proposed  [161].  An  underwater  flexible  robot  manipulation  (called  HEU  Hand  II) 
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that  utilized  PBNNIC  (position-based  neural  network  impedance  control)  for  the  force 
tracking  control  was  studied  in  Ref.  [129]. 

This  work  from  [129]  developed  a  dexterous  underwater  robot  hand,  called  the  HEU 
Hand  II.  The  sensor  system  mainly  includes  12  strain  gauges  at  different  locations.  When 
the  robot  hand  is  under  water,  the  control  system  is  more  complicated  because  the  com¬ 
plete  dynamic  model  is  not  known  exactly.  Hence,  the  control  system  considers  the  uncer¬ 
tainty  of  the  robot  dynamic  model.  The  controller  of  the  hand  force  tracking  is  designed 
by  the  PBNNIC  scheme.  Using  biologically-inspired  principles  for  design  and  control  of 
a  bionic  robot  arm  [106],  several  control  approaches  were  presented  such  as  trajectory 
planning  and  optimization  based  on  robot  dynamics. 

An  alternate  learning  control  strategy  was  proposed  [163]  based  on  the  working 
assumptions  that  both  human  motor  commands  and  sensory  information  are  passed 
on  in  a  discrete,  episodic  manner,  quantized  in  time  with  a  learning  algorithm,  called 
S-learning,  based  on  sequences  arguing  against  the  traditional  control  approaches  due  to 
the  highly  nonlinear  robot's  dynamics  and  large  number  of  degrees  of  freedom. 

In  the  work  in  Ref.  [206],  the  first  prototype  of  a  five-fingered  prosthetic  hand  fitted 
with  only  3  motors  and  achieving  20  degrees  of  freedom  was  described  using  a  new 
"strings  and  springs"  mechanism  and  a  continuous  wavelet  transform  (CWT)  for  extrac¬ 
tion  of  EMG  inputs  for  a  feed-forward,  back-propagation  neural  network  to  recognize 
the  type  of  grip. 

The  work  in  Ref.  [215]  focuses  on  the  control  system  of  the  hand  and  on  the  opti¬ 
mization  of  the  hand  design.  It  proposes  the  control  action  as  proportional  to  the  super¬ 
ficial  EMG  signals  extracted  by  surface  electrodes  applied  to  a  couple  of  antagonistic 
user's  residual  muscles.  This  work  first  explains  designs  of  the  hand  prototype,  such 
as  biomechatronic  design  approach,  under-actuated  artificial  hand,  3D  CAD  model  (by 
ProEngineer)  and  dynamic  analysis  (by  ANSYS).  Second,  it  builds  the  model  of  control 
system,  including  the  kinetics  and  dynamics  of  hand,  in  PD  control  in  the  joint  space 
and  slider  space  with  elastic  compensation.  Third,  it  validates  and  optimizes  the  hand 
design  in  four  multiple  objective  problems  (or  goals).  The  first  two  goals  are  related  to  the 
closed-loop  control  performance  and  the  remaining  two  goals  are  part  of  joint  trajectories. 
It  develops  the  simulation  in  MATLAB/Simulink.  Finally,  it  compares  the  experimental 
results  with  the  simulation. 

The  dynamic  system  of  a  nonlinear  flexible  robot  arm  with  a  tip  mass  was  introduced 
in  Ref.  [198].  Here,  the  proposed  intelligent  optimal  controller,  in  which  the  fuzzy  neural 
network  controller  and  robust  controller  were  designed  to  learn  a  nonlinear  function  and 
compensate  the  approximation  errors  respectively,  could  control  the  coupling  of  bending 
vibration  and  torsional  vibration  for  the  periodic  motion.  To  overcome  the  traditional 
fuzzy  logic  difficulties,  such  as  large  rule  bases  and  long  training  times,  researchers 
[17]  proposed  a  self-learning  dynamic  fuzzy  network  (DFN)  with  dynamic  equality  con¬ 
straints  to  speed  up  the  trajectory  calculations  for  intelligent  nonlinear  optimal  control. 
For  a  five-finger  under-actuated  prosthetic  hand  with  tendon  transmission,  Cipriani  et  al. 
[46]  presented  a  robust  controller  that  implemented  two  subsequent  and  different  phases, 
including  the  pre-shaping  of  the  hand  and  the  involved  fingers  rapidly  closing  around 
the  object. 

2.3.  Main  Control  Technique  Overview  Since  2007 
2.3.1.  Hard  Computing  Techniques 

1.  PD  Controller:  Rong  et  al.  [164]  presented  one  kind  of  PD  controller  with  feed¬ 
forward  control  based  on  adaptive  theory  for  2  DOF  direct  driven  robot  with 
uncertain  parameters. 

2.  Adaptive  Controller:  Cai  et  al.  [27]  developed  an  observer  back-stepping  adaptive 
control  scheme  for  a  two-link  manipulator  under  unmeasured  velocity  and  uncer¬ 
tain  environment.  The  adaptive  velocity  observer  was  designed  independently 
from  the  state-feedback  controller  in  order  to  compensate  the  estimation  errors.  Seo 
and  Akella  [175]  derived  the  novel  adaptive  control  solution  involving  a  new  filter 
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design  for  the  regressor  matrix  for  n  DOF  robot  manipulator  systems.  By  develop¬ 
ing  the  Fourier  series  expansion  from  input  reference  signals  of  every  joint,  Liuzzo 
and  Tomei  [122]  designed  a  global  output  error  feedback  adaptive  learning  control 
for  2  DOF  planar  robot  with  uncertain  dynamics.  To  achieve  the  tracking  control 
objective,  Chen  et  al.  [41]  proposed  an  adaptive  sliding-mode  dynamic  controller 
for  wheeled  mobile  robots  with  system  uncertainties  and  disturbances  to  make  the 
real  velocity  of  the  wheeled  mobile  robot  reach  the  desired  velocity  command. 

3.  Robust  Controller:  Because  of  the  visco-elastic  properties  of  manipulator  links, 
Torabi  and  Jahed  [189]  utilized  the  loop-shaping  method,  which  can  decrease  the 
order  of  the  robust  control  model  of  a  single-link  manipulator  examined  in  time 
and  frequency  domains.  To  enhance  control  of  powered  prosthetic  hands,  Engeberg 
and  Meek  [64-67]  proposed  robust  sliding  mode,  back-stepping,  and  hybrid  slid¬ 
ing  mode-back-stepping  (HSMBS)  parallel  force-velocity  controllers  that  enabled 
the  humans  to  more  easily  control  a  fine  object  by  ten  able-bodied  test  subjects. 
Ziaei  et  al.  [214]  developed  the  modeling,  system  identification  adopting  gener¬ 
alized  orthonormal  basis  functions  (GOBFs),  and  robust  position  and  force  con¬ 
trollers  for  single  flexible  link  (SFL)  manipulators  required  to  operate  the  contact 
motion.  Jiang  and  Ge  [97]  transformed  the  nonlinear  kinematic  models  of  3  DOF 
wheels  mobile  robot  with  uncertain  disturbance  into  linear  control  systems  through 
an  approximate  linearization  algorithm  and  then  designed  a  partial  feedback  H ^ 
robust  controller  through  linear  matrix  inequality  (LMI). 

4.  Optimal  Controller:  Vitiello  et  al.  [196]  synthesized  the  position  controller  and  the 
Kalman  filter  to  perform  the  planar  movements,  such  as  reaching  and  catching, 
of  the  NEURARM  hydraulic  piston  actuation  with  nonlinear  springs  connected  on 
the  cable.  Vrabie  et  al.  [197]  designed  an  online  method  via  a  biological  inspired 
Actor /Critic  structure  to  solve  the  adaptive  optimal  continuous-time  control  prob¬ 
lem  by  the  solution  of  the  algebraic  Riccati  equation  without  using  knowledge  of 
the  system  internal  dynamics.  To  minimize  the  positioning  time  (traveling  between 
two  specific  points)  of  an  under-actuated  2  DOF  robot  manipulator  restricted  to 
the  input  constraint  and  the  structural  parameter  constraint,  Cruz-Villar  et  al.  [52] 
developed  a  concurrent  structure-control  redesign  method  that  combined  the  struc¬ 
tural  parameters  and  a  bang-bang  control  law.  Duchaine  et  al.  [62]  derived  the  posi¬ 
tion  tracking  and  velocity  control,  the  dynamic  model  of  the  robot,  the  prediction 
and  control  horizons,  and  the  constraints  by  a  general  predictive  control  law  and 
also  derived  an  analytical  solution  for  the  optimal  control  by  a  computationally- 
efficient-model-based  predictive  control  scheme  for  a  6  DOF  cable-driven  parallel 
manipulator. 

4.  Hierarchical  Controller:  Fainekos  et  al.  [70]  proposed  a  hierarchical  control  law 
addressing  the  temporal  logic  motion  planning  problem  for  mobile  robots  mod¬ 
eled  by  second  order  dynamics  to  track  a  simpler  kinematic  model  with  a  globally 
bounded  error,  and  then  solved  the  new  robust  temporal  logic  path  planning  prob¬ 
lem  for  the  kinematic  model  using  automata  theory  and  simple  local  vector  fields. 

2.3.2.  Soft  Computing  Techniques 

1.  Fuzzy  Logic:  According  to  the  human  anatomy,  Arslan  et  al.  [13]  developed  the 
biomechanical  model  with  a  tendon  configuration  of  the  3  DOF  index  finger  of  the 
human  hand.  The  fuzzy  sliding  mode  controller  in  which  a  fuzzy  logic  unit  tuned 
the  slope  of  the  sliding  surface  was  introduced  to  generate  the  required  tendon 
forces  during  closing  and  opening  motions. 

2.  Artificial  Neural  Networks:  Onozato  and  Maeda  [149]  utilized  two  neural  net¬ 
works  learning  inverse  kinematic  and  inverse  dynamic  to  control  the  positions  of  2 
DOF  SCARA  robot.  Aggarwal  et  al.  [4]  obtained  the  neural  recordings  from  rhesus 
monkeys  with  three  different  movements,  the  flexion/ extension  of  each  finger,  the 
rotation  of  wrist,  and  dexterous  grasps.  They  also  designed  the  separate  decod¬ 
ing  filters  for  each  movement  by  using  multilayer  feed-forward  artificial  neural 
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network  (ANN)  (implemented  in  real-time  MATLAB/Simulink).  An  online  decen¬ 
tralized  neural  network  control  design  without  deriving  the  dynamic  model  for  a 
class  of  large-scale  uncertain  robot  manipulator  systems  was  proposed  by  Tan  et  al. 
[184].  Kato  et  al.  [104]  expressed  the  reaction  of  brains  to  the  adaptable  prosthetic 
system  for  a  13  DOF  EMG  signal  controlled  prosthetic  hand  with  an  EMG  pattern 
recognition  learning  by  artificial  neural  networks.  In  addition,  functional  magnetic 
resonance  imaging  (f-MRI)  was  used  to  analyze  the  reciprocal  adaptation  between 
the  human  brain  and  the  prosthetic  hand  by  the  plasticity  of  the  motor  and  sensory 
cortex  area  in  brains  based  on  the  variations  in  the  phantom  upper  limb. 

3.  Genetic  Algorithm:  Marcos  et  al.  [54]  proposed  the  closed-loop  pseudo-inverse 
method  with  genetic  algorithms  (CLGA)  to  minimize  the  largest  joint  displacement 
between  two  adjacent  configurations,  the  total  level  of  joint  velocities,  the  joint 
accelerations,  the  total  joint  torque,  and  the  total  joint  power  consumption  for  the 
trajectory  planning  of  3  DOF  redundant  robots.  Kamikawa  and  Maeno  [98]  used  a 
genetic  algorithm  to  optimize  locations  of  pivots  and  grasping  force  and  designed 
one  ultrasonic  motor  to  move  15  compliant  joints  for  an  under-actuated  five-finger 
prosthetic  hand. 

2.3.3.  Hybrid  of  Soft  and  Hard  Computing  Techniques 

1.  PID  Controller  and  Robust  Controller:  Dieulot  and  Colas  [61]  presented  a  case 
study  of  the  design  of  robust  parametric  methods  for  flexible  axes  and  an  heuristic 
initial  tuning  of  the  PID  controller  from  additional  pole  placement  constraints  on 
the  rigid  mode. 

2.  Adaptive  Controller  and  Robust  Controller:  To  implement  the  trajectory  tracking 
mission  under  the  influence  of  unknown  friction  and  uncertainty,  Chen  et  al.  [40] 
utilized  a  composite  tracking  scheme,  including  the  adaptive  friction  estimation  to 
determine  Coulomb  friction,  viscous  friction,  and  the  Stribeck  effect  and  a  robust 
controller  to  enhance  the  overall  stability  and  robustness  for  a  2  DOF  planar  robot 
manipulator. 

3.  Robust  Controller  and  Optimal  Controller:  Huang  et  al.  [87]  designed  the  robust 
control  systems  with  some  uncertainties,  including  the  unknown  payload  and 
unknown  modeling  of  objects,  and  the  unknown  dynamic  parameters,  such  as  the 
performance  index  that  was  optimized  by  the  optimal  control  method  for  the  space 
robot  to  capture  unknown  objects. 

4.  Robust  Controller  and  Fuzzy  Logic:  Tootoonchi  et  al.  [188]  combined  a  robust 
quantitative  feedback  theory  (QFT)  designed  to  follow  the  desired  trajectory  track¬ 
ing  with  the  fuzzy  logic  controller  (FLC)  designed  to  reduce  the  complexities  of 
the  system  dynamics  for  2  DOF  arm  manipulator.  The  control  gain  of  the  sliding 
mode  controller  tuned  according  to  error  states  of  the  system  by  a  fuzzy  controller 
and  a  moving  sliding  surface  whose  slope  is  dynamically  changed  by  a  fuzzy  logic 
algorithm  for  a  3  DOF  spatial  robot  were  presented  by  Yagiz  and  Hacioglu  [204]. 

5.  Robust  Controller  and  Artificial  Neural  Networks:  Siqueira  and  Terra  [180]  devel¬ 
oped  a  neural  network-based  H ^  controller  that  approximated  the  uncertain  factors 
of  an  actual  under-actuated  cooperative  manipulator.  They  robustly  controlled  the 
position  and  squeeze  force  errors  between  the  manipulator  end-effectors  and  the 
object,  although  one  joint  was  not  actuated. 

6.  Sliding  Mode  Controller  and  Genetic  Algorithm:  Chen  and  Chang  [42]  utilized 
the  multiple-crossover  genetic  algorithm  to  estimate  the  unknown  system  param¬ 
eters  and  the  sliding  mode  control  method  to  overcome  the  uncertainty  for  a  two- 
link  robot  control. 

7.  Sliding  Mode  Controller  and  Particle  Swarm  Optimization:  Salehi  et  al.  [166] 
used  an  online  particle  swarm  optimization  to  tune  the  parameters  of  sliding  mode 
control  at  the  contact  moments  of  end  effector  and  unknown  environments  for  the 
2  DOF  planar  manipulator. 

8.  Fuzzy  Logic  and  Artificial  Neural  Networks:  Subudhi  and  Morris  [183]  proposed 
a  hybrid  fuzzy  neural  control  (HFNC)  scheme  containing  a  fuzzy  logic  controller 
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and  a  neural  network  controller  to  balance  the  coupling  effects  for  the  multi-link 
flexible  manipulator  with  both  rigid  and  flexible  motions. 

9.  Artificial  Neural  Networks  and  Particle  Swarm  Optimization:  Wen  et  al.  [202] 
addressed  the  hybrid  particle  swarm  optimization  neural  network  (HPSONN)  to 
compute  the  pseudo-inverse  Jacobian  of  2  DOF  planar  manipulator  inverse  kine¬ 
matic  control. 


2.4.  Revolutionary  Prosthesis 

Recently  (see  the  press  releases  [7,  58])  the  Applied  Physics  Laboratory  (APL)  of  Johns 
Hopkins  University  (JHU)  in  Baltomore,  MD,  USA  received  funding  for  the  Revolution¬ 
ary  Prosthesis  2009  program  from  DARPA  (Defense  Advanced  Research  Projects  Agency), 
the  U.S.  Department  of  Defense,  to  " develop  a  next-generation  mechanical  arm  that 
mimics  the  properties  and  sensory  perception  of  the  real  thing."  The  APL  leads  an  inter¬ 
national  team  of  about  30  organizations  from  Austria,  Canada,  Germany,  Italy,  Sweden, 
and  USA.  The  APL  team  delivered  its  first  DARPA  Limb  Proto  1  (see  Ref.  [8],  which  "is  a 
complete  limb  system  that  also  includes  a  virtual  environment  used  for  patient  training, 
clinical  configuration,  and  to  record  limb  movements  and  control  signals  during  clinical 
investigations"). 

3.  HYBRID  INTELLIGENT  CONTROL  STRATEGIES 

Here  we  present  the  recent  research  activities  on  hybrid  control  strategies  for  a  smart 
prosthetic  hand  conducted  at  Idaho  State  University  (ISU).  The  schematic  diagram  of 
the  work  is  shown  in  Figure  1  (see  Refs.  [120,  160,  212]).  The  overall  system,  in  brief, 
consists  of  electromyographic  (EMG)  signal  acquisition  from  the  user  arm  for  surface 
or  implanted  electrodes  (in  the  implanted  case  we  focus  on  biocompatibility  based  on 
nano-materials  research).  The  EMG  signal  is  then  processed  for  feature  extraction  and 
classification  or  identification  of  EMG  signal  to  correspond  to  different  motions  of  the 
prosthetic  hand.  The  classified  signal  is  then  used  to  control  the  prosthetic  hand  using 
actuators  and  driving  mechanisms.  It  is  to  be  noted  in  this  proposal  that  the  EMG  signal 
extraction  and  identification  and  the  control  algorithm  are  investigated  using  the  fusion 
of  soft  computing  (SC)  and  hard  computing  (HC)  techniques. 

3.1.  Fusion  of  Soft  and  Hard  Computing  Techniques 

Soft  computing  (SC)  or  computational  intelligence  (Cl)  is  an  emerging  field  based  on 
synergy  and  seamless  integration  of  neural  networks  (NN),  fuzzy  logic  (FL),  and  genetic 
algorithms  (GA)  [102].  The  previous  works  on  prosthetic  hands  used  artificial  neural 
networks  [45,  88,  120],  fuzzy  logic  [5,  31,  200],  genetic  algorithms  [73],  etc.  mostly  for 
EMG  signal  classification  for  various  movements  or  functions  of  the  prosthetic  hand. 

In  hybrid  intelligent  control  strategies,  we  look  at  the  integrated  structure  by  blend¬ 
ing  [153,  186]  soft  computing  (SC)  techniques  and  conventional  hard  computing  (HC) 


Figure  1.  Schematic  diagram  of  prosthetic  hand  technology. 
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techniques  comprising  optimal  control  [140],  robust  control  [213],  nonlinear  control  [110] 
and/or  adaptive  control  [185]  with  specific  applications  to  prosthetics. 

The  integration  of  SC  and  HC  methodologies  shown  in  Figure  2  has  the  following 
attractive  features  [153,  186]: 

1.  The  methodology  based  on  SC  can  be  used,  in  particular  with  fuzzy  logic,  at  upper 
levels  of  the  overall  mission  where  human  involvement  and  decision  making  is  of 
primary  importance.  Whereas,  the  HC  can  be  used  at  lower  levels  for  accuracy, 
precision,  stability  and  robustness. 

2.  In  another  situation  using  a  hybrid  scheme,  a  neural  network  of  the  SC  is  used 
to  supplement  the  control  provided  by  a  linear,  fixed  gain  controller  for  a  missile 
autopilot. 

3.  Further,  the  SC-based  GA  can  be  used  for  parameter  tuning  of  a  PID  controller  to 
achieve  good  performance  and  robustness  for  a  wide  range  of  operating  conditions. 

4.  The  SC  and  HC  are  potentially  complementary  methodologies. 

5.  The  fusion  could  solve  problems  that  cannot  be  solved  satisfactorily  by  using  either 
methodology  alone. 

6.  Novel  synergetic  combinations  of  SC  and  HC  lead  to  high  performance,  robust, 
autonomous,  and  cost-effective  missions. 

The  research  at  Idaho  State  University  (ISU)  focuses  on  developing  intelligent 
autonomous  strategies  for  EMG  signal  extraction,  analysis,  and  control  of  prosthetics  by 
fusion  of  soft  computing  techniques  comprising  NN,  FL,  and  GA  [141,  153]  and  hard 
computing  (HC)  techniques.  The  proposal  takes  advantage  of  our  in-house  research  expe¬ 
rience  with  problems  in  prosthetics  [20,  63]  in  particular,  and  with  problems  in  biomedical 
engineering  [142,  143]  in  general. 

An  overview  of  nine  papers  was  presented  using  the  techniques  in  industrial  and  engi¬ 
neering  applications  [152].  For  the  fusion  techniques.  Sick  and  Ovaska  [179]  described  a 
multidimensional  categorization  scheme  in  five  aspects:  the  degree  of  interconnection  of 
soft  and  hard  computing  components  (fusion  grade),  the  topology  of  fusion  skills  (fusion 
structure),  the  time  when  fusion  happens  (fusion  time),  the  layer  of  a  system  architecture 
(fusion  level),  and  the  motivation  for  the  application  (fusion  incentive).  Further,  Ovaska 
et  al.  [151]  classified  the  fusion  techniques  into  twelve  main  categories  and  six  supple¬ 
mentary  categories. 


Figure  2.  Principle  behind  fusion  of  soft  and  hard  computing. 


Page  296  of  531 


16 


Automatic  Control  Techniques  for  Smart  Prosthetic  Hand  Technology:  An  Overview 


4.  OVERVIEW  OF  RESEARCH  AT  ISU 

A  chronological  overview  of  the  research  at  Idaho  State  University  is  provided  below. 

A  short  review  by  Lai  et  al.  [115]  notes  the  importance  of  the  biological  interfaces 
between  robotic  implants  and  other  prosthetic  devices  and  notes  an  interdisciplinary 
team  of  biomedical  and  tissue  engineers,  and  biomaterial  and  biomedical  scientists  is 
needed  to  work  together  holistically  and  synergistically. 

In  addressing  the  particle  swarm  optimization  technique,  a  set  of  operators  for  a  Parti¬ 
cle  Swarm  Optimization  (PSO)-based  algorithm  is  investigated  for  the  purpose  of  finding 
optimal  values  for  some  of  the  classical  benchmark  problems.  Particle  swarm  algorithms 
are  implemented  as  mathematical  operators  inspired  by  the  social  behaviors  of  bird  flocks 
and  fish  schools.  In  addition,  particle  swarm  algorithms  utilize  a  small  number  of  rela¬ 
tively  less  complicated  rules  in  response  to  complex  behaviors,  such  that  they  are  compu¬ 
tationally  inexpensive  in  terms  of  memory  requirements  and  processing  time.  In  particle 
swarm  algorithms,  particles  in  a  conpf^fous  variable  space  are  linked  with  neighbors, 
therefore  the  updated  velocity jneans^^Jparticles  influence  the  simulation  results.  The 
work  presents  a  statistical  investigation  on  the  velocity  update  rule  for  continuous  vari¬ 
able  PS  algorithm.  In  particular,  the  probability  density  function  influencing  the  particle 
velocity  update  is  investigated  along  with  the  components  used  to  construct  the  updated 
velocity  vector  of  each  particle  within  a  flock.  The  simulation  results  of  several  numerical 
benchmark  examples  indicate  that  a  small  amount  of  negative  velocity  is  necessary  to 
obtain  good  optimal  values  near  global  optimality  [32]. 

A  chronological  overview  of  the  applications  of  control  theory  to  prosthetic  hand  is 
presented  by  Naidu  et  al.  [139]  focusing  on  hard  computing  or  control  techniques,  such 
as  multi-variable  feedback,  optimal,  nonlinear,  adaptive  and  robust,  and  soft  computing 
or  control  techniques,  such  as  artificial  intelligence,  neural  networks,  fuzzy  logic,  genetic 
algorithms,  particle  swarm,  and  on  the  fusion  of  hard  and  soft  control  techniques.  The 
work  [34]  presents  the  PSO  algorithm  for  identifying  the  rupture  force  for  leukocyte 
adhesion  molecules  and  the  problem  of  finding  the  correct  control  parameters  of  a  robotic 
hand.  Other  work  at  ISU  is  the  fusion  of  soft  computing  or  control  technique  of  genetic 
algorithm  (GA)  and  hard  computing  technique  of  proportional  integral  derivative  (PID) 
control  with  application  to  prosthetic  hand.  In  particular,  an  adaptive  neuro-fuzzy  infer¬ 
ence  system  (ANFIS)  is  used  for  inverse  kinematics  of  the  three-link  index  finger,  and 
feedback  linearization  is  used  for  the  dynamics  of  the  hand.  The  GA  is  used  to  find  the 
optimal  parameters  of  the  PID  controller  [35].  An  adaptive  PSO  (APSO)  approach  based 
on  altering  the  maximum  velocity  at  each  iteration  for  two  thirty-dimensional  benchmark 
problems  is  used  [33]. 

A  hybrid  of  a  soft  computing  technique  of  adaptive  neuro-fuzzy  inference  system 
(ANFIS)  and  a  hard  computing  technique  of  adaptive  control  for  a  two-dimensional 
movement  of  a  prosthetic  hand  with  a  thumb  and  index  finger  was  investigated  [38]. 
The  results  of  this  hybrid  controller,  when  compared  with  the  PID  controller  showed 
enhanced  performance.  A  novel  condensed  hybrid  optimization  (CHO)  algorithm  using 
enhanced  continuous  tabu  search  (ECTS)  and  the  PSO  was  examined  [39].  The  proposed 
CHO  algorithm  combines  the  respective  strengths  of  ECTS  and  the  PSO.  In  particular, 
the  ECTS  is  utilized  to  define  smaller  search  spaces  that  are  used  in  a  second  stage  by 
the  basic  PSO  to  find  the  respective  local  optimum.  The  ECTS  covers  the  global  search 
space  by  using  a  TS  concept  called  diversification  and  then  selects  the  most  promising 
areas  in  the  search  space.  Once  the  promising  regions  in  the  search  space  are  defined, 
the  proposed  CHO  algorithm  employs  another  TS  concept  called  intensification  in  order 
to  search  the  promising  area  thoroughly.  The  proposed  CHO  algorithm  is  tested  with 
the  multi-dimensional  Hyperbolic  and  Rosenbrock  problems.  Compared  to  the  other  four 
algorithms,  the  results  indicate  that  the  accuracy  and  effectiveness  of  the  proposed  CHO 
algorithm  were  enhanced.  Another  hybrid  of  a  soft  computing  technique  using  the  ANFIS 
and  a  hard  computing  or  control  technique  using  finite-time  linear  quadratic  optimal 
control  for  a  two-fingered  (thumb  and  index)  prosthetic  hand  was  investigated  [37].  In 
particular,  the  ANFIS  is  used  for  inverse  kinematics,  and  the  optimal  control  is  used 
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to  minimize  tracking  errors  utilizing  feedback  linearized  dynamics.  The  simulations  of 
this  hybrid  controller,  when  compared  with  the  proportional-integral-derivative  (PID) 
controller,  showed  enhanced  performance.  This  work  was  extended  to  a  five-fingered, 
three-dimensional  prosthetic  hand  [36]. 


5.  RECENT  DEVELOPMENTS  IN  NEUROPROSTHETICS 

It  is  worth  noting  some  of  the  recent  developments  in  neuroprosthesis  [85,  84,  144,  176]. 

An  interesting  study  was  made  by  Hart  et  al.  [80]  on  implanted  neuroprostheses 
employing  functional  electric  simulation  (FES)  to  provide  grasp  and  release  to  individ¬ 
uals  with  tetraplegia  and  comparing  three  control  methods  for  shoulder  position,  wrist 
position,  and  myoelectric  wrist  extensors.  To  improve  the  control  of  grasp  strength,  fore¬ 
arm  pronation,  and  elbow  extension  to  people  with  spinal  cord  injury  at  C5  and  C6, 
the  investigation  by  Peckham  et  al.  [155]  developed  an  advanced  neuroprosthesis  that 
includes  implanted  components,  including  a  10-channel  stimulator,  leads,  electrodes,  and 
a  joint  angle  transducer,  and  external  components,  such  as  a  control  unit  and  transmitter- 
receiver  coil. 

A  particularly  interesting  case  was  reported  [48,  78,  150]  regarding  Jesse  Sullivan,  who 
lost  both  arms  in  an  electrical  accident.  He  could  move  his  bionic  arm  with  his  brain — 
basically  rewiring  the  severed  live  nerves  that  control  arm  and  hand  movements  by 
redirecting  the  nerves  to  pectoral  muscles  in  his  chest.  Electrodes  attached  to  the  chest 
muscles  produce  an  electrical  signal  that  controls  the  robotic  arm  depending  upon  the 
nature  of  muscle  movement,  which  in  itself  is  characterized  by  " thinking"  in  the  brain 
what  is  to  be  done  with  arms.  However,  the  demonstrated  bionic  arm  is  only  a  "prototype 
and  for  research  only." 

More  interesting  news  appeared  [47,  82]  regarding  implantation  of  an  electronic  chip 
into  the  brain  of  a  quadripledge  man  to  use  a  computer  to  operate  a  robotic  arm. 

6.  FUTURE  DIRECTIONS 

During  the  last  three  decades,  investigations  have  been  carried  out  on  the  use  of  EMG 
signals  to  develop  a  prosthetic  hand  to  perform  as  many  functions  as  possible.  How¬ 
ever,  the  use  of  EMG  signals  is  limited  in  the  number  of  possible  human-like  functions 
with  as  few  electrodes  as  possible  and  also  for  the  prosthetic  hand  to  ultimately  have 
a  natural  "cosmetic"  appearance.  Further,  the  EMG  signal  cannot  provide  any  kind  of 
feedback  to  the  user  [207].  One  of  several  possible  solutions  to  overcome  the  limitations 
of  surface-based  EMG  approach  is  neuroprosthesis.  Here,  we  use  an  interface  between 
the  peripheral  nervous  system  (PNS)  and  the  "natural"  neural  interface  to  extract,  record, 
and  simulate  the  PNS  in  a  selective  way.  Further,  advances  in  biocompatible  neural  inter¬ 
faces  can  provide  some  sensory  feedback  to  the  user  by  stimulating  the  afferent  nerves 
and  allowing  motor  control  of  the  prosthesis  leading  to  a  "natural"  EMG-based  control. 
Based  on  this,  two  possible  ways  of  controlling  prosthetic  hands  are  the  simple  and  non- 
intrusive  EMG-based  control  and  the  more  complicated,  implantable  EMG-based  control. 

In  a  recent  conference  of  the  National  Academies  Keck  Futures  Initiative,  "Smart  Pros¬ 
thetics:  Exploring  Assistive  Devices  for  Body  and  Mind,"  [12],  it  was  pointed  out,  "We 
can  make  smarter  prostheses  when  we  are  smarter  in  integrating  state-of-the-art  neuro¬ 
science  with  state-of-the-art  engineering,  medicine,  and  social  science." 
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Abstract :  An  adaptive  control  strategy  for  the  14  degrees  of  freedom  (DOFs),  five-fingered  smart  prosthetic  hand 
with  unknown  mass  and  inertia  of  all  the  fingers  is  developed  in  this  work.  In  modeling,  the  various  links  used 
for  the  five  fingers  of  the  prosthetic  hand  are  shown.  A  cubic  polynomial  for  the  trajectory  planning  is  used.  In 
particular,  using  a  desired  orientation  for  three-link  fingers,  the  forward  and  inverse  kinematics  of  the  prosthetic 
hand  system  regarding  the  analytical  solutions  between  the  angular  positions  of  joints  and  the  positions  and  ori¬ 
entations  of  the  end-effectors  (fingertips)  have  been  obtained.  The  simulations  of  the  resulting  adaptive  controller 
with  five-fingered  prosthetic  hand  show  enhanced  performance. 

Key-Words:  adaptive  control,  prosthetic  hand,  hard  control,  five  finger  hand,  feedback  linearization,  trajectory 
planning 


1  Introduction 

Due  to  the  extreme  complexity  of  human  hand  that 
has  27  bones,  controlled  by  about  38  muscles  to  pro¬ 
vide  the  hand  with  22  degrees  of  freedom  (DOFs), 
and  incorporates  about  17,000  tactile  units  of  4  dif¬ 
ferent  units,  reproducing  the  human  hand  in  all  its 
various  functions  and  appearance  is  still  a  challeng¬ 
ing  task  [1].  Prosthetic  hands  have  been  built  to  re¬ 
place  human  hands  that  can  fully  operate  the  various 
motions,  such  as  holding,  moving,  grasping,  lifting, 
twisting  and  so  on  [1-6].  However,  about  35%  of  the 
users  do  not  regularly  use  their  prosthetic  hands  be¬ 
cause  of  several  reasons,  including  poor  functionality 
of  the  presently  available  prosthetic  hands  and  psy¬ 
chological  problems.  Thus,  designing  and  developing 
an  artificial  hand  which  can  “mimic  the  human  hand 
as  closely  as  possible”  both  in  functionality  and  ap¬ 
pearance  can  overcome  these  problems. 

Hard  computing/control  (HC)  techniques  are  used 
at  lower  levels  for  accuracy,  precision,  stability 
and  robustness.  HC  comprises  proportional-integral 
(PI)  control  [7],  proportional-integral-derivative  (PID) 
control  [8-10],  optimal  control  [11,12],  adaptive  con¬ 
trol  [13-16],  robust  control  [17],  etc.  with  specific  ap¬ 
plications  to  prosthetic  devices.  However,  our  previ¬ 
ous  work  [15]  for  a  two-fingered  (thumb  and  index  fin¬ 
ger)  prosthetic  hand  showed  that  adaptive  controller 


can  overcome  overshooting  and  oscillation.  However, 
a  five-fingered  prosthetic  hand  with  adaptive  control 
technique  has  not  been  developed  yet. 

In  this  work,  we  first  describe  briefly  the  trajec¬ 
tory  planning  problem,  human  hand  anatomy  and  the 
inverse  kinematics  for  two-link  thumb  and  the  remain¬ 
ing  three-link  fingers  (index,  middle,  ring  and  little). 
Next,  the  dynamics  of  the  prosthetic  hand  is  derived 
and  feedback  linearization  technique  is  used  to  ob¬ 
tain  linear  tracking  error  dynamics.  Then  the  adap¬ 
tive  controller  is  designed  to  minimize  the  tracking  er¬ 
ror.  The  simulation  results  show  that  the  five-fingered 
prosthetic  hand  with  the  proposed  adaptive  controller 
can  grasp  an  object  without  overshooting  and  oscilla¬ 
tion.  The  last  section  provides  conclusions  and  future 
work. 


2  Modeling 

2.1  Human  Hand  Anatomy 

Figure  1  (a)  shows  a  normal  human  hand  composed  of 
thumb  (£),  index  (i),  middle  (m),  ring  (r),  little  (/)  fin¬ 
gers  and  palm.  The  wrist  is  located  between  the  fore¬ 
arm  and  the  hand  and  consists  of  eight  carpal  bones 
organized  in  two  rows  of  proximal  (movable)  and  dis¬ 
tal  (immovable)  carpal  bones  as  shown  in  Figure  1  (b) 
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l:  Thumb 
i:  Index  Finger 
wt:  Middle  Finger 
r.  Ring  Finger 
f:  Liole  Finger 


1 


4, 


(a)  Physical  Appearance  of  Right  Hand 
(Anterior  View) 


Carpels 

{proximal  row) 


S:  Scaphoid  bone 
L  :  Lunate  bone 
T\  Triquetrum  bone 
P:  Pisiform  bone 


T:  Trapezium  bone 
T:  Trapezoid.  bone  Carpali 
C:  Capstale  bone  (distal  row) 
H\  Hamate  bone 


ing  surfaces  between  two  bones  and  can  be  classi¬ 
fied  as  saddle  joints.  Index,  middle,  ring,  and  lit¬ 
tle  fingers  include  three  revolute  joints  in  order  to  do 
the  angular  movements  (Figure  1  (b)).  Metacarpal- 
phalangeal  (MCP)  joint  is  located  between  metacarpal 
and  proximal  phalange  bones;  proximal  and  dis¬ 
tal  interphalangeal  (PIP  and  DIP)  joints  separate 
the  phalangeal  bones.  Thumb  contains  metacarpal- 
phalangeal  (MCP)  and  interphalangeal  (IP)  joints 
[18].  In  this  work,  q{,  q2  and  qJ3  (J  =  i,  m,  r  and  l ) 
represent  the  angular  positions  (or  joint  angles)  of  the 
first  joint  MCPf  the  second  joint  PIPi  and  the  third 
joint  DIPi  of  index,  middle,  ring  and  little  fingers,  re¬ 
spectively;  q\  and  q2  are  the  angular  positions  of  the 
first  joint  MCP1  and  the  second  joint  IP1  of  thumb  (£), 
respectively. 


Carpali 


Mctaearpals 


Proximal  phalanges 


Middle  phalanges 


Discjl  phalanges 


Mldtuipj]  Joint 
CMC  Jainl 


MCP  Joint 


PI  P  Joint 
DIP  Joint 


(bj  Bones  of  Left  Hand  {Posterior  View) 


Figure  1:  Human  Wrist  and  Hand:  (a)  Physical  Ap¬ 
pearance  of  Right  Hand  (Anterior  View)  (b)  Bones  of 
Left  Hand  (Posterior  View) 


2.2  Trajectory  Planning 

The  trajectory  planning  using  cubic  polynomial  for 
fingertip  position  control  was  discussed  in  our  previ¬ 
ous  work  [5,  8, 15, 19, 20]  for  a  two-fingered  (thumb 
and  index  finger)  smart  prosthetic  hand.  For  three- 
link  fingers,  we  present  the  same  technique  for  finger¬ 
tip  orientation  control.  A  time  history  of  desired  (d) 
fingertip  orientation  (</>)  and  its  differentiation  (</>  and 


(j>)  is  given  as 

m 

—  CJO  +  t  +  (jJ2 t^  +  ^3^, 

(1) 

m 

=  (jj\  T  2uj2t  3a;3t2, 

(2) 

pm 

=  2cJ2  +  6cJ3£, 

(3) 

[18].  The  proximal  row  (top)  of  carpal  bones  from  lat¬ 
eral  to  medial  is  the  Scaphoid ,  Lunate ,  Triquetrum  and 
Pisiform ;  the  distal  row  (bottom)  of  carpal  bones  from 
medial  to  lateral  has  the  Hamate ,  Capitate ,  Trape¬ 
zoid  and  Trapezium.  The  hand  is  composed  of  five 
Metacarpals  and  five  Digits.  The  metacarpals  pro¬ 
duce  a  curve,  so  the  palm  is  concave  in  the  resting  po¬ 
sition.  The  five  digits  contain  one  thumb  (t)  and  four 
fingers,  e.g.  index  (z),  middle  (m),  ring  (r),  and  lit¬ 
tle  (/)  fingers,  respectively.  The  thumb  has  two  bones, 
Proximal  phalanx  and  Distal  phalanx.  Each  fin¬ 
ger  consists  of  three  bones,  Proximal  phalanx ,  Mid¬ 
dle  phalanx  and  Distal  phalanx.  In  this  work,  we  as¬ 
sumed  that  the  palm  is  fixed,  the  thumb  has  two  links 
(proximal  phalanx  and  distal  phalanx),  and  each  fin¬ 
ger  has  three  links  (proximal  phalanx,  middle  phalanx 
and  distal  phalanx). 

Synovial  joints  are  formed  at  the  surface  of  rela¬ 
tive  motion  between  two  bones.  The  joints  of  thumb 
and  four  fingers  contain  two  saddle-shaped  articulat- 


where  cjo-^3  are  undetermined  constants  and  the  su¬ 
perscript  j  indicates  the  index  of  each  finger  (j  =  i, 
m,  r  and  l).  The  relations  (1)  and  (2)  need  to  sat¬ 
isfy  the  constraint  conditions  at  initial  time  to  and  final 
time  tf.  This  can  be  written  as 


Tfl  =  $. 

Here,  the  matrices  T,  12,  and  are 


(4) 


■  i 

to 

f1 

l0 

4  ' 

T  = 

0 

1 

1 

tf 

2£o 

t2 

lf 

3  4 

tf 

5 

(5) 

_  0 

1 

2  tf 

3 tf. 

o 

3 

II 

£ 

L02 

<^3  ] 

/ 

5 

(6) 

$  = 

>o 

1  1 

/ 

(7) 

,  the  4  unknown  constants,  cjq-^3, 

can  be 

computed  by  12  =  T  1$. 
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2.3  Kinematics 

Kinematics  is  the  study  of  geometry  in  motion  and 
is  restricted  to  a  natural  geometrical  description  of 
motion  by  the  manners,  including  positions,  orienta¬ 
tions,  and  their  derivatives  (velocities  and  accelera¬ 
tions)  [21,22].  Forward  and  inverse  kinematics  of 
articulated  systems  study  the  analytical  relationship 
between  the  angular  positions  of  joints  and  the  posi¬ 
tions  and  orientations  of  the  end-effectors  (fingertips). 
A  desired  trajectory  is  usually  specified  in  Cartesian 
space  and  the  trajectory  controller  is  easily  performed 
in  the  joint  space.  Hence,  conversion  of  Cartesian  tra¬ 
jectory  planning  to  the  joint  space  [22]  is  necessary. 
Using  inverse  kinematics,  the  joint  angular  positions 
of  each  finger  need  to  be  obtained  from  the  known  fin¬ 
gertip  positions.  Then  the  angular  velocities  and  an¬ 
gular  accelerations  of  joints  can  be  obtained  from  the 
linear  and  angular  velocities  and  accelerations  of  fin¬ 
gertips  (end-effectors)  by  differential  kinematics.  The 
inverse  and  differential  kinematics  of  two-link  thumb 
and  three-link  fingers  were  discussed  in  our  previous 
publications  [5,8, 15, 19,20]  for  a  two-fingered  (thumb 
and  index  finger)  smart  prosthetic  hand. 

For  five  fingers  shown  in  Figure  2,  XG,  YG ,  and 


2.4  Dynamics  of  Hand 

The  dynamic  equations  of  hand  motion  are  derived  via 
Lagrangian  approach  using  kinetic  energy  and  poten¬ 
tial  energy  as  [5] 


d_  fdC\  _dC_ 
dt\d  q)  dq  T 


(8) 


where  C  is  the  Lagrangian;  q  and  q  represent  the  an¬ 
gular  velocities  and  angle  vectors  of  joints,  respec¬ 
tively;  t  is  the  given  torque  vector  at  joints.  The  La¬ 
grangian  C  can  be  expressed  as 


C  =  T-V , 


(9) 


where  T  and  V  denote  kinetic  and  potential  energies, 
respectively.  Substituting  (9)  into  (8),  dynamic  equa¬ 
tions  of  thumb  can  be  obtained  as  below. 


M(q)q  +  C(q,  q)  +  G(q)  =  r ,  (10) 

where  M(q)  is  the  inertia  matrix;  C(q,  q)  is  the  Cori¬ 
olis/centripetal  vector  and  G(q)  is  the  gravity  vector. 
(10)  can  be  also  written  as 

M(q)q  +  N(q,  q)  —  r,  (11) 

where  N(q,  q)  =  C(q,  q)  +  G(q)  represents  non¬ 
linear  terms.  The  dynamic  relations  for  the  two-link 
thumb  and  the  remaining  three-link  fingers  are  quite 
lengthy  and  omitted  here  due  to  lack  of  space  [5]. 


3  Control  Techniques 

3.1  Feedback  Linearization 

The  nonlinear  dynamics  represented  by  (11)  is  to 
be  converted  into  a  linear  state-variable  system  us¬ 
ing  feedback  linearization  technique  [14].  Alterna¬ 
tive  state-space  equations  of  the  dynamics  can  be  ob¬ 
tained  by  defining  the  position/velocity  state  x(£)  of 
the  joints  as 


Figure  2:  The  Relationship  between  Global  Coordi¬ 
nate  and  Local  Coordinates 

ZG  are  the  three  axes  of  the  global  coordinate.  The 
local  coordinate  xt-yt-zt  of  thumb  can  be  reached  by 
rotating  through  angles  a  and  (3  to  XG  and  Y G  of  the 
global  coordinate,  subsequently.  The  local  coordinate 
xl-yl-zl  of  index  finger  can  be  obtained  by  rotating 
through  angle  a  to  XG  and  then  translating  the  vector 
d1  of  the  global  coordinate;  similarly,  the  local  coor¬ 
dinate  xi-yi-z^  of  middle  finger  (j  —  m),  ring  finger 
(j  =  r),  and  little  finger  ( j  =  l)  can  be  obtained  by  ro¬ 
tating  through  angle  a  to  XG  and  then  translating  the 
vector  d-i  (j  =  m,  r  and  l )  of  the  global  coordinate. 


x(f)  =  [q'(t)  q'(i)]'.  (12) 

Let  us  repeat  the  dynamical  model  and  rewrite  (1 1)  as 

|q (0  =  -M-'(qM)  [N(q(t),  q(())  -  T (()] .  (13) 

Thus,  from  (12)  and  (13),  we  can  derive  a  linear  sys¬ 
tem  in  Brunovsky  canonical  form  as 


x(t)  = 


0  I 
0  0 


x(*)  + 


(14) 


with  its  control  input  vector  given  by 
U (i)  =  — M_1(q (t))  [N(q(t),  q(t))  -  r(t)} .  (15) 
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Let  us  suppose  the  prosthetic  hand  is  required  to  track 
the  desired  trajectory  q d(t)  described  under  path  gen¬ 
eration  or  tracking.  Then,  the  tracking  error  e(t)  is 
defined  as 

e(t)  =  qa(t)  -  q(i).  (16) 

Here,  q d(t)  is  the  desired  angle  vector  of  joints  and 
can  be  obtained  by  trajectory  planning  [5,8, 15, 19,20]; 
q (£)  is  the  actual  angle  vector  of  joints.  Differentiat¬ 
ing  (16)  twice,  to  get 

e(i)  =  q d(t)  ~  q(f),  e(f)  =  qd(f)  -  q(f).  (17) 

Substituting  (13)  into  (17)  yields 

e(t)  =  qd(t)  +M~1(q(t))  [N(q(f),  q(f))  -  r(f)]  (18) 


Figure  3:  Block  Diagram  of  Adaptive  Control  Tech¬ 
nique 


from  which  the  control  function  can  be  defined  as 

U (t)  =  qd(t)  +M~1(q(t))  [N(q(t),q(t))  -  r(f)] .  (19) 


where  C(q(t),  q(£))  =  Cm(q(t),  q(t))q(t)  and  the  non¬ 
linear  term  f  (t)  can  be  defined  as 


This  is  often  called  the  feedback  linearization  control  law, 
which  can  also  be  inverted  to  express  it  as 

r(f)  =  M(q(t))  [qd(t)  -  u(f))  +  N(q(f),  q(f)] .  (20) 


f(f)  =  M(q(f))(qd(f)  + Ae(f))  +  G(q(f))  + 

Cm(q{t),q(t))(qd(t)  +  Ae(f))  +  Tdis, 

=  Ytt.  (26) 


Using  the  relations  (17)  and  (19),  and  defining  state  vector 
x(£)  =  [e'(f)  e'(t)]',  the  tracking  error  dynamics  can  be 
written  as 


x(f) 


0  I 
0  0 


x(f)  + 


(21) 


Here,  t(us  is  the  unknown  disturbance.  Y  is  a  regression 
matrix  of  known  robot  functions  and  7r  is  a  vector  of  un¬ 
known  parameters  [13].  The  regression  matrix  Y  and  the 
unknown  parameter  vector  7r  of  two-link  thumb  and  three- 
link  index  finger  are  given  in  Appendix  [5].  The  torque 
vector  r  (t)  can  be  calculated  by 


Note  that  this  is  in  the  form  of  a  linear  system  such  as 


r(t)  =  f(t)  +  KDr(t).  (27) 


x(t)  =  A  x(t)  +  Bu(f).  (22) 

3.2  Adaptive  Control  Technique 

The  tracking  error  e(t)  and  the  filtered  tracking  error  r(t) 
are  defined  as 

e(f)  =qd(t)  -q(f),  (23) 

r  (t)  =  e(t)  +  A  e(t).  (24) 

Here,  q  d(t)  is  the  desired  angle  vector  of  joints;  q  (t)  is  the 
actual  angle  vector  of  joints;  A  =  diag(  Ai,  A2, . . .,  An)  is 
the  positive-definite  diagonal  gain  matrix.  The  filtered  er¬ 
ror  (24)  ensures  stability  of  the  overall  system  so  that  the 
tracking  error  (23)  is  bounded.  Figure  3  shows  the  block 
diagram  of  the  adaptive  controller.  Here,  the  filtered  sig¬ 
nal  r (t)  is  derived  from  the  tracking  error  e(t)  and  the  tra¬ 
jectory  planner  and  is  fed  to  the  adaptive  controller  of  the 
prosthetic  hand.  Differentiating  and  substituting  (24)  into 
(10)  gives  the  dynamic  equation  in  terms  of  the  filtered  er¬ 
ror  r  (t)  as 

M(q(£))r(£)  =  -Cm(q(t),  q(t))r(t)  +  f  (f)  -  r(t),  (25) 


The  unknown  parameter  rate  vector  7r  can  be  updated  by 

7 t  =  r_1Y'r(i)  (28) 

where  T  is  a  tuning  parameter  diagonal  matrix. 

4  Simulation  Results  and  Discussion 

Figure  4  shows  that  five-fingered  prosthetic  hand  with  14 
DOFs  is  reaching  a  rectangular  rod  in  order  to  grasp  the  ob¬ 
ject.  When  thumb  and  the  other  four  fingers  are  performing 
extension/flexion  movements,  the  workspace  of  fingertips 
is  restricted  to  the  maximum  angles  of  joints.  Referring  to 
inverse  kinematics,  the  first  and  second  joint  angles  of  the 
thumb  fingertip  are  constrained  in  the  ranges  of  [0,90]  and 
[-80,0]  (degrees).  The  first,  second,  and  third  joint  angles 
of  the  other  four  fingers  are  constrained  in  the  ranges  of 
[0,90],  [0,110]  and  [0,80]  (degrees),  respectively  [23]. 

Next,  we  present  simulations  with  an  adaptive  con¬ 
troller  for  the  14  DOFs  five-fingered  smart  prosthetic  hand. 
The  parameters  of  the  two-link  thumb/three-link  fingers 
[24]  were  related  to  desired  trajectory.  All  parameters  of 
the  smart  prosthetic  hand  selected  for  the  simulations  are 
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Table  1 :  Parameter  Selection  of  the  Smart  Hand 


Figure  4:  Five-Finger  Prosthetic  Hand  Reaching  a 
Rectangular  Rod 

given  in  Table  1  and  the  side  length  and  length  of  the  tar¬ 
get  rectangular  rod  are  0.010  and  0.100  (m),  respectively. 
All  initial  actual  angles  are  zero.  The  relating  parameters 
between  the  global  coordinate  and  the  local  coordinates  are 
defined  in  Table  2. 

Besides,  in  this  work,  we  assumed  that  each  link  of  all 
fingers  is  a  circular  cylinder  with  the  radius  ( R )  0.010  (m), 
so  the  inertia  IJzzk  of  each  link  k  of  all  fingers  j  (=  t,  i,  m, 
r  and  l )  can  be  calculated  as 

l3zzk  =  \miR2  +  \mkL3k-  (29) 

Figure  5,  Figure  7,  Figure  9,  Figure  11  and  Figure  13 
show  the  tracking  errors  of  thumb,  index,  middle,  ring,  and 
little  fingers  for  the  proposed  five-fingered  smart  prosthetic 
hand,  respectively.  Figure  6,  Figure  8,  Figure  10,  Figure  12 
and  Figure  14  show  the  desired/actual  angles  of  thumb,  in¬ 
dex,  middle,  ring,  and  little  fingers  for  the  proposed  five¬ 
fingered  smart  prosthetic  hand,  respectively.  The  observa¬ 
tion  that  all  tracking  errors  dramatically  drop  within  one 
second  and  are  less  than  one  degree  after  convergence  pro¬ 
vides  the  evidence  that  the  adaptive  controller  for  the  14- 
DOFs  prosthetic  hand  enhances  performance.  The  other 
observation  that  after  convergence,  all  three-link  fingers 
show  more  unstable  errors  than  two-link  thumb  suggests 
that  the  more  DOFs  increase  the  difficulty  of  the  adaptive 
controller  without  knowing  the  mass  and  inertia  of  the  links 
of  all  fingers. 

5  Conclusions  and  Future  Work 

An  adaptive  control  strategy  was  developed  for  the  14  de¬ 
grees  of  freedom  (DOFs),  five-fingered  smart  prosthetic 
hand  with  unknown  mass  and  inertia  of  all  the  fingers. 
Further,  the  forward  and  inverse  kinematics  of  the  sys¬ 
tem  regarding  the  analytical  relationship  between  the  an¬ 
gular  positions  of  joints  and  the  positions  and  orientations 


Parameters 

Values 

Thumb 

Time  (t0,tf)* 

0,  20  (sec) 

Desired  Initial  Position** 

0.035,  0.060  (m) 

Desired  Final  Position** 

0.0495,  0.060  (m) 

Desired  Initial  Velocity* 

0,  0  (m/s) 

Desired  Final  Velocity* 

0,  0  (m/s) 

Length 

0.04,  0.04  (m) 

Index  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.010,  0.060  (m) 

Desired  Initial  </>o*** 

75  (deg) 

Desired  Final  </>/*** 

160  (deg) 

Desired  Initial  </>o*** 

0  (m/s) 

Desired  Final  </>/*** 

0  (m/s) 

Length 

0.04,  0.04,  0.03  (m) 

Middle  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.005,  0.060  (m) 

Length 

0.04,  0.04,  0.03  (m) 

Ring  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.010,  0.060  (m) 

Length 

0.04,  0.04,  0.03  (m) 

Little  Finger 

Desired  Initial  Position** 

0.055,  0.080  (m) 

Desired  Final  Position** 

0.020,  0.060  (m) 

Length 

0.04,  0.04,  0.03  (m) 

*A11  fingers  use  same  parameters 
**Local  coordinates 

***A11 3-link  fingers  use  same  parameters 


of  the  end-effectors  (fingertips),  was  obtained  using  a  de¬ 
sired  orientation  for  three-link  fingers.  The  simulations  of 
the  resulting  adaptive  controller  showed  good  agreement 
between  the  reference  and  the  actual  trajectories.  Work  is 
in  progress  for  developing  an  adaptive/robust  controller  for 
the  five  fingered  hand  with  14-DOFs. 
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U.S.  Department  of  the  Army,  under  the  award  number 
W81XWH-10-1-0128  awarded  and  administered  by  the 
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Table  2:  Parameter  Selection  of  the  Conversion  from 
Global  to  Local  Coordinates 


Parameters 

Values 

Rotating  cr 

90  (deg) 

Rotating  (3 

45  (deg) 

Translating  d1 

(0.035, 0, 0)  (m) 

Translating  dm 

(0.040, 0,  -0.020)  (m) 

Translating  dr 

(0.035,0,  -0.040)  (m) 

Translating  d1 

(0.025,0,  -0.060)  (m) 

100 - ■ - ■ - 

80  ■  —  Joint  1  of  Thumb 

—  Joint  2  of  Thumb 


-20  ■ 

-40  ■ 

-60 - - - - - - - 

0  5  10  15  20 

Time  (sec) 


Figure  5:  Tracking  Errors  of  Adaptive  Controller  for 
Two-Link  Thumb 


sociation  proceedings,  etc. 

Appendix:  Regression  Matrix  Y  and  Unknown  Param¬ 
eter  Vector  7 r 

In  Section  3.2,  the  regression  matrix  Yt  and  the  unknown 
parameter  vector  7rt  of  two-link  thumb  can  be  expressed  as 


Figure  6:  Tracking  Angles  of  Adaptive  Controller  for 
Two-Link  Thumb 

L/2  =  (2L\lt2C2  +  L\L\  +  44)(to  +  Aiei) 

+  (L\l\C2  +  44)  (#<22  +  ^2  £2) 

— Ti  4  *^2^2  (to  +  Aiei) 

—Lt1lt2S2((ll  +  #2X342  +  ^2^2)  +  #4^12, 

L/3  =  (jdi  +  Aiei, 

Xl4  =  #di  +  Aiei  +  qd2  +  ^2  &2, 

_  yt  _  n 

F22  =  (Li4c2  +  44)(<M  +  Aiei)  +  44(te  +  A2e2) 

+Li4*S2#i  (#<21  +  Aiei) 

^£l4$2#l(#d2  +  A2e2)  +  gl\Cv2, 

Y24  =  Qdl  H-  Aiei  +  qd2  +  A2e2, 

Cl  =  cos(#i),  C2  =  cos(#2 )  5 
#2  =  sin(#2),  C12  =  cos(#i  +  #2). 

Similarly,  the  regression  matrix  Yl  and  the  unknown 
parameter  vector  7 r2  of  three-link  index  finger  can  be  writ¬ 
ten  as 


II 

-K> 

'  n. 1 

n 2 

Yl 3 

*?4  ' 

n 1 

n 2 

^3 

^4  J 

ll 

-K> 

k 

m\  m2  r. \zl 

^2] 

r. 

r  v* 

vi 

*12 

yi 

*13 

yi 

*14 

yi 

*13 

yi 

*13 

11 

yi 

*21 

Yi 

*22 

yi 

*23 

yi 

*24 

yi 

*23 

yi 

*26 

Yi 

L  J31 

yi 

*32 

yi 

*33 

yi 

*34 

yi 

*33 

yi 

*36 

TV1  =  [m\  ml  m\  Pzzl  Pzz2  Pzz3]' . 


where 


Here, 


Y\i  —  44  (to  +  Aiei)  +  gl\C\, 


Y\i  —  l\l\(qdi  +  Aiei)  +  gl\Cx  +  #4Ci2, 
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Figure  7:  Tracking  Errors  of  Adaptive  Controller  for  Figure  8:  Tracking  Angles  of  Adaptive  Controller  for 
Three-Link  Index  Finger  Three-Link  Index  Finger 


Yl 2  =  (2L\li2S1S12  +  2L\li2C1C12  +  L\L\  +  l\l\){qdl  +  Aid) 

+  (+146l6i2  +  +F2C1.612  +  44)  (4d2  +  ^2&2)  +  gLiCi 
+(+{461612  -  Li4Ci5i2)«iWd2  +  A2e2) 

+(+{461612  -  l\ACxS^)q^qdi  +  Aiei) 

+(+{461612  -  LAClSi2)q2{qd2  +  A2e2), 


F13  =  (2LiLi5i5i2  +  2L\L\CiCi2  +  2+{zj6i6i23 

+244CiCi23  +  2+^6126123  +  244C12C123 

+44  +  +2+2  +  44X4+  +  Aid) 

+  (2+246l26l23  +  2L2/3C12C123  +  L1L2S1S12 
++{+{61612  +  L[liSlSl2S  +  44ClCl23 
++{4  +  44)  (4+  +  A2d) 

+  (+146l6l23  +  44C1C123  +  +X36126123 

+  L2/3C12C123  +  44)  (4+  +  Asd) 

+g+i6i  +  gL<2  C\2  +  ghCi23 

+(+{+{61612  —  +1+{6i6i2)4i(4c*2  +  A2e2) 
+(+{+{61612  —  +{+{61612)42  {qdi  +  Aiei) 

+  (+146l6l23  —  44ClSl23)<h  (4^2  +  A2e2) 
+(+{4616123  —  +{4616123)42(4^1  +  Aid) 

+  (+{4616123  —  L\r3Ci  6123)41  (4^3  +  Ases) 
+(+{4616123  —  L\r3CiSi23)q3(qdi  +  Aid) 

+  (+{£36126123  ~  +{46l26l23)4l(4d3  +  Ases) 

+  (+{£36126123  ~  +{  4  Cl  2  5^123  )  ^3  (4gU  +  Aid) 

+  (+{£3616123  —  44^6123)^2(^3  +  A3e3) 

+  (446iCi23  —  L\l3Ci  6123)^3(^2  +  A2e2) 


+  (+{46126123  —  +{4  Ci  2  6123)42  (<?c£3  +  Ases) 

+  (446l2Cl23  —  44Cl2  6123)^3(^2  +  A2e2) 

+  (+{+{61612  —  44Cl6l2)(/2(gd2  +  A2e2) 

+  (446lCl23  —  44Cl6l23)42(4d2  +  A2e2) 

+  (446lCl23  —  44616123)43(4^3  +  A3e3) 

+(446126123  —  +{46126123)43(4^3  +  Ases), 

4.4  =  Qdi  +  Aid, 

4s  =  to  H~  Aid  +  to  +  A24, 

4e  =  to  +  44  +  44  +  A24  +  44  +  Asd, 

_ v*  _ v*  _ v*  _ v*  _ n 

*21  —  *24  —  *31  “  *32  —  *34  —  *35  ~  u> 

4 2  =  (446i6i2  +  446i6i2  +  44)(4di  +  A1e1) 

+44  (to  +  A24)  +  ghCi2 

+(4461612  -  4461612)42(4^1  +  Aid) 
+(4461612  -  4446i2)4i(4di  +  Aid), 

43  —  (2446126123  +  2446126123  +  4461612 

+4461612  +  44616123  +  44616123 
+44  +  44)  (4+  +  Aid)  +  (2446126123 
+2446126123  +  44  +  44X4+2  +  A2d) 

+  (+246126123  +  +246126123  +  4  4)  (to  +  A3d) 

-\-gL2C12  +  ghC\23 

+  (+{+{61612  -  +i  +{61 612)41  (4*2  +  A2e2) 

+  (+146l6l23  —  +146l6l23)4l(4d2  +  A2e2) 
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Tracking  Errors  (deg) 


Figure  9:  Tracking  Errors  of  Adaptive  Controller  for 
Three-Link  Middle  Finger 


Figure  10:  Tracking  Angles  of  Adaptive  Controller 
for  Three-Link  Middle  Finger 


+  (£2/36126123  —  £2/36126123)^1(^3  +  Ases) 

+  (£2/36126123  —  £2/36126123)^3(^1  +  Aid) 

+  (£2/36126123  —  £2/36126123)^2(^3  +  Ases) 

+  (£2/36126123  —  £2/36126123)^3(^2  +  A2e2) 

+  (£16261612  —  £1  £261612)^1(^1  +  Aid) 

+  (£1/3616123  —  £1/3616123)^1(^1  +  Aid) 

+  (£2/36126123  —  £2/36126123)^3(^3  +  Ases), 

^25  =  Qdi  +  Aid  +  qd2  +  A2e2, 

=  Qdi  +  Aid  +  qd2  X2 ^2  +  qd3  +  Asd, 

Li  =  (£1/3616123  +  £i46i6i23  +  446126123 

+  £2/36126123  +  ^3){qdl  +  Aid) 

+  (£2/36126123  +  £2/36126123  +  44)  (qd2  +  A2d) 
+4/3  (<Zd3  +  ^3^3)  +  9I3  6123 
+  (£2/36126123  —  £2/36126123)^1(^2  +  A2e2) 

+  (£2/36126123  —  £2/36126123)^2(^1  +  Aid) 

+  (£1/3616123  —  £1/3616123)^1(^3  +  Ases) 

+  (£2/36126123  —  £2/36126123)^1(^3  +  Ases) 

+  (£1/36126123  —  £2/36126123)^2(^3  +  Ases) 
+(£1/3616123  —  £1/3616123)^1(^1  +  Aid) 

+  (£146126123  —  £1/36126123)^1(^1  +  Aid) 

+  (£146126123  —  £1/36126123)^2(^2  +  A2e2), 

F36  =  qdi  +  Aid  +  qd2  +  A2d  +  qd3  +  A3  £3, 


Ci  =  cos(d),  C12  =  cos (q1  +  g2), 
61  =  sin(d),Si2  =  sin(gjH-g2), 

6123  =  cos(d +  ^2  +  ^3), 

6123  =  sin(d  +  <?2  +  £3)- 
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Abstract:  In  this  paper,  an  adaptive  control  strategy  is  developed  for  the  14  degrees  of  freedom  (DOFs),  five¬ 
fingered  smart  prosthetic  hand  with  unknown  mass  and  inertia  of  all  the  fingers.  In  particular,  the  forward  and 
inverse  kinematics  of  the  system  regarding  the  analytical  relationship  between  the  angular  positions  of  joints  and 
the  positions  and  orientations  of  the  end-effectors  (fingertips)  have  been  obtained  using  a  desired  orientation  for 
three-link  fingers.  The  simulations  of  the  resulting  adaptive  controller  with  five-fingered  prosthetic  hand  show 
enhanced  performance. 

Key-Words :  adaptive  control,  prosthetic  hand,  hard  control,  five  finger  hand,  feedback  linearization,  trajectory 
planning 


1  Introduction 

Due  to  the  extreme  complexity  of  human  hand,  that 
has  27  bones,  controlled  by  about  38  muscles  to  pro¬ 
vide  the  hand  with  22  degrees  of  freedom  (DOFs), 
and  incorporates  about  17,000  tactile  units  of  4  dif¬ 
ferent  units,  reproducing  the  human  hand  in  all  its 
various  functions  and  appearance  is  still  a  challeng¬ 
ing  task  [1].  Prosthetic  hands  have  been  built  to  re¬ 
place  human  hands  that  can  fully  operate  the  various 
motions,  such  as  holding,  moving,  grasping,  lifting, 
twisting  and  so  on  [1-5].  However,  about  35%  of  the 
users  do  not  regularly  use  their  prosthetic  hands  be¬ 
cause  of  several  reasons,  including  poor  functionality 
of  the  presently  available  prosthetic  hands  and  psy¬ 
chological  problems.  Thus,  designing  and  developing 
an  artificial  hand  which  can  “mimics  the  human  hand 
as  closely  as  possible”  both  in  functionality  and  ap¬ 
pearance  can  overcome  these  problems. 

Hard  computing/control  (HC)  techniques  can  be 
used  at  lower  levels  for  accuracy,  precision,  stability 
and  robustness.  HC  comprises  proportional-integral- 
derivative  (PID)  control  [6],  optimal  control  [7,  8], 
adaptive  control  [9-11]  etc.  with  specific  applica¬ 
tions  to  prosthetic  devices.  However,  our  previous 
work  [11]  for  a  two-fingered,  thumb  and  index  fin¬ 
ger,  prosthetic  hand  showed  that  adaptive  controller 
can  overcome  overshooting  and  oscillation.  However, 
a  five-fingered  prosthetic  hand  with  adaptive  control 
technique  has  not  been  developed  yet. 

In  this  work,  we  first  describe  briefly  the  trajec¬ 


tory  planning  problem,  human  hand  anatomy  and  the 
inverse  kinematics  for  two-link  thumb  and  the  remain¬ 
ing  three-link  fingers  (index,  middle,  ring  and  little). 
Next,  the  dynamics  of  the  prosthetic  hand  is  derived 
and  feedback  linearization  technique  is  used  to  ob¬ 
tain  linear  tracking  error  dynamics.  Then  the  adap¬ 
tive  controller  is  designed  to  minimize  the  tracking  er¬ 
ror.  The  simulation  results  show  that  the  five-fingered 
prosthetic  hand  with  the  presented  adaptive  controller 
can  grasp  an  object  without  overshooting  and  oscilla¬ 
tion.  Conclusions  and  future  work  are  provided  in  the 
last  section. 


2  Modeling 

2.1  Trajectory  Planning  and  Inverse  Kine¬ 
matics 

The  trajectory  planning  using  cubic  polynomial  was 
discussed  in  our  previous  work  [5,  6,  11,  12]  for  a 
two-fingered  (thumb  and  index  finger)  smart  pros¬ 
thetic  hand.  Figure  1  shows  that  index  finger,  mid¬ 
dle  finer,  ring  finer  and  little  finger  contain  three 
revolute  joints  in  order  to  do  the  angular  move¬ 
ments.  Metacarpal-phalangeal  (MCP)  joint  is  lo¬ 
cated  between  metacarpal  and  proximal  phalange 
bone;  proximal  and  distal  interphalangeal  (PIP  and 
DIP)  joints  separate  the  phalangeal  bones.  Thumb 
contains  metacarpal-phalangeal  (MCP)  and  interpha¬ 
langeal  (IP)  joints.  In  this  work,  q[ ,  qJ2  and  q3  repre- 
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Figure  1:  The  Joints  of  Five-Finger  Prosthetic  Hand 
Reaching  a  Rectangular  Rod 


Figure  2:  The  Definition  of  Global  Coordinate  and 
Local  Coordinates 


sent  the  angular  positions  (or  joint  angles)  of  the  first 
joint  MCPi ,  the  second  joint  and  the  third 

joint  DIPi  of  index  finger  (j  =  i),  middle  finger 
(j  =  m),  ring  finger  (j  —  r )  and  little  finger  (j  =  Z), 
respectively;  q\  and  q\  are  the  angular  positions  of  the 
first  joint  MCP 1  and  the  second  joint  IP 1  of  thumb 
(t\ 

Forward  and  inverse  kinematics  of  articulated 
systems  study  the  analytical  relationship  between  the 
angular  positions  of  joints  and  the  positions  and  orien¬ 
tations  of  the  end-effectors  (fingertips).  A  desired  tra¬ 
jectory  is  usually  specified  in  Cartesian  space  and  the 
trajectory  controller  is  easily  performed  in  the  joint 
space.  Hence,  to  convert  Cartesian  trajectory  plan¬ 
ning  to  the  joint  space  [13]  is  necessary.  Using  inverse 
kinematics,  the  joint  angular  positions  of  each  finger 
need  to  be  obtained  from  the  known  fingertip  positions 
( joint  space).  Then  the  angular  velocities  and  angular 
accelerations  of  joints  can  be  obtained  from  the  linear 
and  angular  velocities  and  accelerations  of  fingertips 
(end-effectors)  by  the  geometric  Jocobian.  As  shown 
in  Figure  2,  XG ,  YG ,  and  ZG  are  the  three  axes  of 
the  global  coordinate.  The  local  coordinate  xt-yt-zt 
of  the  thumb  can  be  reached  by  rotating  through  an¬ 
gles  a  and  (3  to  XG  and  YG  of  the  global  coordinate, 
subsequently.  The  local  coordinate  xl-yl-zl  of  index 
finger  can  be  obtained  by  rotating  through  angle  a  to 
XG  and  then  translating  the  vector  d1  of  the  global 
coordinate;  similarly,  the  local  coordinate  x^-yi-z^  of 
middle  finger  (j  —  m),  ring  finger  (j  =  r),  and  little 
finger  (j  —  l)  can  be  obtained  by  rotating  through  an¬ 
gle  a  to  XG  and  then  translating  the  vector  (j  =  m, 
r  and  l)  of  the  global  coordinate.  The  inverse  kine¬ 
matics  of  two-link  thumb  and  three-link  fingers  was 
discussed  in  our  previous  publications  [5,6, 1 1, 12]  for 


a  two-fingered  (thumb  and  index  finger)  smart  pros¬ 
thetic  hand. 


2.2  Dynamics  of  Hand 

The  dynamic  equations  of  hand  motion  are  derived  via 
Lagrangian  approach  using  kinetic  energy  and  poten¬ 
tial  energy  as  [5] 


d_  fdC\  _  PC 
dt  \  dq  J  dq 


(1) 


where  C  is  the  Lagrangian;  q  and  q  represent  the 
angular  velocity  and  angle  vectors  of  joints,  respec¬ 
tively;  t  is  the  given  torque  vector  at  joints.  The  La¬ 
grangian  C  can  be  expressed  as 


C  =  T-V , 


(2) 


where  T  and  V  denote  kinetic  and  potential  energies, 
respectively.  Substitute  (2)  into  (1)  and  dynamic  equa¬ 
tions  of  thumb  can  be  obtained  as  below. 


M(q)q+C(q,q) +  G(q)  =  r,  (3) 

where  M(q)  is  the  inertia  matrix;  C(q,  q)  is  the  Cori¬ 
olis/centripetal  vector  and  G(q)  is  the  gravity  vector. 
(3)  can  be  also  written  as 

M(q)q  +  N(q,  q)  —  r,  (4) 

where  N(q,  q)=C(q,  q)+G(q)  represents  nonlinear 
terms.  The  dynamic  relations  for  the  two-link  thumb 
and  the  remaining  three-link  fingers  are  quite  lengthy 
and  omitted  here  due  to  lack  of  space  [5]. 
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3  Control  Techniques 


3.2  Adaptive  Control  Technique 


3.1  Feedback  Linearization 

The  nonlinear  dynamics  represented  by  (4)  is  to 
be  converted  into  a  linear  state-variable  system  by 
finding  a  transformation  using  feedback  linearization 
technique  [10].  Alternative  state-space  equations  of 
the  dynamics  can  be  obtained  by  defining  the  posi¬ 
tion/velocity  state  x(£)  of  the  joints  as 

x(t)  =  [q(t)'  q(i)']'.  (5) 

Let  us  repeat  the  dynamical  model  and  rewrite  (4)  as 


(*)  =  -M(q (t))  1  [N(q(t),  q(i))  -  r(t)] .  (6) 


Thus,  from  (5)  and  (6),  we  can  derive  a  linear  state- 
variable  equation  in  Brunovsky  canonical  form  as 


±(t)  = 


0  I 
0  0 


x(*)  + 


(7) 


with  its  control  input  vector  given  by 

U (t)  =  -M(q(i))_1  [N(q(t),  q(t))  -  r(t)} .  (8) 


Let  us  suppose  the  prosthetic  hand  is  required  to  track 
the  desired  trajectory  q d(t)  described  under  path  gen¬ 
eration  or  tracking.  Then,  the  tracking  error  e(t)  is 
defined  as 


e(t)  =  -  q(0-  (9) 

Here,  q d(t)  is  the  desired  angle  vector  of  joints  and 
can  be  obtained  by  trajectory  planning  [5, 6, 11, 12]; 
q (t)  is  the  actual  angle  vector  of  joints.  Differentiat¬ 
ing  (9)  twice,  to  get 

e(i)  =  qd(i)  -  q(*),  e(f)  =  qd(f)  -  q(t).  (10) 

Substituting  (6)  into  (10)  yields 

e(f)  =  qd(t)  +M(q(f))_1  [N(q(f),  q(f))  -  r(f)]  (11) 

from  which  the  control  function  can  be  defined  as 

U (t)  =  q d(t)  +  M(q(f))_1  [N(q(f),  q(t))  -  r(f)] .  (12) 

This  is  often  called  the  feedback  linearization  control 
law,  which  can  also  be  inverted  to  express  it  as 

r(f)  =  M(q(t))  [q d(t)  -  u(t))  +  N(q(f),  q(f)] .  (13) 


Using  the  relations  (10)  and  (12),  and  defining  state 
vector  x(£)  [e(t)'  e^)']',  the  tracking  error  dy¬ 

namics  can  be  written  as 


x(t)  = 


0  I 
0  0 


x(*)  + 


0 

I 


u  (*)• 


(14) 


Note  that  this  is  in  the  form  of  a  linear  system  such  as 


The  tracking  error  e  and  the  filtered  tracking  error  r 
are  defined  as 


e  =  qd-q,  (16) 

r  =  e  +  Ae.  (17) 

Here,  q d  is  the  desired  angle  vector  of  joints;  q  is  the 
actual  angle  vector  of  joints;  A  is  the  positive-definite 
diagonal  gain  matrix.  The  filtered  error  (17)  ensures 
stability  of  the  overall  system  so  that  the  tracking  error 
(16)  is  bounded.  Figure  3  shows  the  block  diagram  of 
the  adaptive  controller.  Here,  the  filtered  signal  r  (t) 
is  derived  from  the  tracking  error  e(t)  and  the  trajec¬ 
tory  planner  and  is  fed  to  the  adaptive  controller  of  the 
prosthetic  hand. 


Figure  3:  Block  Diagram  of  Adaptive  Control  Tech¬ 
nique 

Differentiating  and  substituting  (17)  into  (3)  gives 
the  dynamic  equation  in  terms  of  the  filtered  error  r  as 

M(q)r=  — Cm(q,  q)r  +  f  — r  (18) 

where  C(q,  q)  =  Cm(q,  q)q  and  the  nonlinear  term  f 
can  be  defined  as 

f  =  M(q)(qd  + Ae) +  Cm(q,q)(qd  + Ae) 
+G(q)  +  Tdis 

=  Ytt.  (19) 

Here,  the  regression  matrix  Y  is  a  matrix  of  known 
robot  functions  and  7r  is  a  vector  of  unknown  param¬ 
eters  [9].  The  regression  matrix  Y  and  the  unknown 
parameter  vector  7r  of  two-link  thumb  and  three-link 
index  finger  are  expressed  in  [5].  The  torque  vector  r 
can  be  calculated  by 


x(£)  =  Ax(f)  +  Bu(f).  (15) 


t  —  Y7T  +  Kor.  (20) 
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The  unknown  parameter  rate  vector  -h  can  be  updated 
by 

7T  =  r_1Y'r  (21) 

where  T  is  a  tuning  parameter  diagonal  matrix. 

4  Simulation  Results  and  Discussion 

When  thumb  and  the  other  four  fingers  are  doing  ex¬ 
tension/flexion  movements,  the  workspace  of  finger¬ 
tips  is  restricted  to  the  maximum  angles  of  joints.  Re¬ 
ferring  to  inverse  kinematics,  the  first  and  second  joint 
angles  of  the  thumb  fingertip  are  constrained  in  the 
ranges  of  [0,90]  and  [-80,0]  (degrees).  The  first,  sec¬ 
ond,  and  third  joint  angles  of  the  other  four  fingers  are 
constrained  in  the  ranges  of  [0,90],  [0,1 10]  and  [0,80] 
(degrees),  respectively  [14].  Next,  we  present  simula¬ 
tions  with  an  adaptive  controller  for  the  14  DOFs  five¬ 
fingered  smart  prosthetic  hand.  The  parameters  of  the 
two-link  thumb/three-link  index  finger  [15]  were  re¬ 
lated  to  desired  trajectory.  All  parameters  of  the  smart 
prosthetic  hand  selected  for  the  simulations  are  given 
in  Table  1  and  the  side  length  and  length  of  the  target 
rectangular  rod  are  0.010  and  0.100  (m),  respectively. 
The  relating  parameters  between  the  global  coordinate 
and  the  local  coordinates  are  defined  in  Table  2.  Be¬ 
sides,  all  links  are  assumed  as  a  circular  cylinder  with 
the  radius  ( R )  0.010  (m),  so  the  inertia  IJzzk  of  each 
link  k  of  each  finger  j  (j  =  t ,  i,  m,  r ,  and  l)  can  be 
calculated  as 

Tizk  =  \miR2  +  lmlLl2-  (22> 

All  initial  actual  angles  are  zero. 

Figure  4  to  Figure  8  are  the  tracking  errors  and 
desired/actual  angles  of  thumb,  index  finger,  middle 
finger,  ring  finger,  and  little  finger  for  the  proposed 
five-fingered  smart  prosthetic  hand. 

5  Conclusions  and  Future  Work 

An  adaptive  control  strategy  was  developed  for  the  14 
degrees  of  freedom  (DOFs),  five-fingered  smart  pros¬ 
thetic  hand  with  unknown  mass  and  inertia  of  all  the 
fingers.  Further,  the  forward  and  inverse  kinematics 
of  the  system  regarding  the  analytical  relationship  be¬ 
tween  the  angular  positions  of  joints  and  the  positions 
and  orientations  of  the  end-effectors  (fingertips),  was 
obtained  using  a  desired  orientation  for  three-link  fin¬ 
gers.  The  simulations  of  for  the  resulting  adaptive 
controller  with  five-fingered  prosthetic  hand  showed 
good  agreement  between  the  reference  and  the  actual 
trajectories.  Work  is  in  progress  for  developing  an 


Table  1 :  Parameter  Selection  of  the  Smart  Hand 


Parameters 

Values 

Thumb 

Time  C tQ,tf )* 

0,  20  (sec) 

Desired  Initial  Position** 

0.035,  0.060  (m) 

Desired  Final  Position** 

0.0495,  0.060  (m) 

Desired  Initial  Velocity* 

0,  0  (m/s) 

Desired  Final  Velocity* 

0,  0  (m/s) 

Length 

0.04,  0.04  (m) 

Index  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.010,  0.060  (m) 

Length 

0.04,  0.04, 0.03  (m) 

Middle  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.005,  0.060  (m) 

Length 

0.04,  0.04, 0.03  (m) 

Ring  Finger 

Desired  Initial  Position** 

0.065,  0.080  (m) 

Desired  Final  Position** 

0.010,  0.060  (m) 

Length 

0.04,  0.04, 0.03  (m) 

Little  Finger 

Desired  Initial  Position** 

0.055,  0.080  (m) 

Desired  Final  Position** 

0.020,  0.060  (m) 

Length 

0.04,  0.04, 0.03  (m) 

*A11  fingers  use  same  parameters 
**Local  coordinates 


adaptive/robust  controller  for  the  five  fingered  hand 
with  14-DOFs. 
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ABSTRACT 

This  paper  addresses  the  optimal  control  for  a  two-fingered 
(thumb  and  index  finger)  prosthetic  hand.  To  speed  up  the 
performance  and  improve  the  accuracy,  we  modified  the 
performance  index  by  including  an  exponential  term.  First, 
we  developed  the  differential  kinematics  for  a  serial  n-link 
revolute-joint  planar  manipulator  to  computer  the  angular 
velocities  and  angular  accelerations  of  each  finger.  Further, 
the  trajectory  planning  of  the  orientation  was  designed  by 
using  cubic  polynomial.  Simulations  show  that  the  pro¬ 
posed  technique  provides  fast  action  with  high  accuracy 
and  30-fold  faster  than  ANFIS  or  GA  based  trajectory  plan¬ 
ning. 

KEY  WORDS 

Prosthetic  Hand,  Optimal  Control,  Stability,  ANFIS 

1  Introduction 

Hard  control  (HC)  techniques  comprise  proportional- 
integral-derivative  (PID)  control  [1-3],  optimal  control 
[4-6],  adaptive  control  [7-9]  etc.  with  specific  applications 
to  prosthetic  devices.  However,  our  previous  works  [2, 3] 
for  smart  prosthetic  hand  showed  that  PID  controller  re¬ 
sults  in  overshooting  and  oscillation.  Optimal  controller 
can  obtain  a  high  performance  [5,6].  However,  this  control 
technique  takes  time  to  be  convergent.  To  overcome  the 
overshooting  and  convergent  problems,  studying  the  stabil¬ 
ity  of  the  optimal  controller  is  precisely  the  main  goal  of 
this  work.  In  addition,  the  adaptive  neuro-fuzzy  inference 
system  (ANFIS)  and  genetic  algorithm  (GA)  based  trajec¬ 
tory  planning  [5, 6, 10]  take  much  time  to  train  the  neural 
layers,  especially  for  high  degree-of-freedom  (DOF)  prob¬ 
lems.  To  improve  the  performance,  it  is  also  the  goal  of  this 
paper. 

In  this  paper,  we  first  consider  briefly  forward  kine¬ 
matics  and  inverse  kinematics  for  a  serial  n- link  revolute- 
joint  planar  manipulator  and  two-link  thumb  and  three-link 
index  finger,  respectively.  Then,  by  using  differential  kine¬ 
matics,  the  angular  velocities  and  angular  accelerations  of  a 
serial  n- link  revolute-joint  planar  manipulator  are  derived. 
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Moreover,  the  trajectory  planning  of  the  orientation  is  de¬ 
signed  by  using  cubic  polynomial.  The  dynamic  models  of 
the  smart  prosthetic  hand  and  linearization  technique  were 
briefly  introduced.  Finally,  the  stability  of  finite-time  linear 
quadratic  optimal  controller  are  investigated  by  changing 
cost  function  J.  The  simulation  results  show  the  presented 
technique  can  be  fast  convergence  and  hold  the  high  per¬ 
formance  for  the  two-fingered  smart  prosthetic  hand. 


2  Modeling 

Forward  and  inverse  kinematics  of  articulated  systems 
study  the  analytical  relationship  between  the  angular  po¬ 
sitions  of  joints  and  the  positions  and  orientations  of  the 
end-effectors  (fingertips).  A  desired  trajectory  is  usually 
specified  in  Cartesian  space  and  the  trajectory  controller 
is  easily  performed  in  the  joint  space.  Hence,  conversion 
of  Cartesian  trajectory  planning  to  the  joint  space  [11]  is 
necessary.  Using  inverse  kinematics,  the  joint  angular  po¬ 
sitions  of  each  finger  need  to  be  obtained  from  the  known 
fingertip  positions.  Then  the  angular  velocities  and  angu¬ 
lar  accelerations  of  joints  can  be  obtained  from  the  linear 
and  angular  velocities  and  accelerations  of  fingertips  (end- 
effectors)  by  differential  kinematics.  The  inverse  kine¬ 
matics  of  two-link  thumb  and  three-link  fingers  were  dis¬ 
cussed  in  our  previous  publications  [2,3,6,10,12]  for  a  two¬ 
fingered  (thumb  and  index  finger)  smart  prosthetic  hand. 
Here,  we  derive  the  general  form  of  forward  and  differen¬ 
tial  kinematics  for  a  serial  n-link  revolute-joint  planar  ma¬ 
nipulator. 


2.1  Forward  Kinematics 

Figure  1  shows  the  illustration  of  a  serial  n- link  revolute- 
joint  planar  manipulator.  Li  is  the  length  of  the  link  i  and  qi 
is  the  angle  of  the  joint  i  (i  =  1,  2,  •  •  • ,  n).  ti  is  the  actuator 
of  the  joint  i.  The  position  Pn  (Xn,  Yn)  of  the  end-effector 
and  the  orientation  <j>n  of  the  end-effector  frame  have  been 
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as 


L\cos(q\)  +  L\cos(q{  +qt2), 
L\  sin(q\)  +  L^sinfal  +qt2). 


(2) 

(3) 


Figure  1 .  Illustration  of  a  Serial  n-Link  Revolute- Joint  Pla¬ 
nar  Manipulator 


derived  [12]  as 


X1  = 
Yt  = 


Here  L\  and  L\  are  the  lengths  of  the  links  1  and  2  of  the 
thumb,  respectively;  q\  and  q2  are  the  angular  positions  (or 
called  angles)  of  joints  1  and  2  of  the  thumb.  Choosing  the 
elbow  up  configuration,  the  angle  q2  of  the  joint  2  can  be 
obtained  from  [11] 


q2  =  —  cos  1 


' Xt2  +  Yt2  -  L\2  -  Ll2' 

2LfZ| 


(4) 


Notice  that  in  this  paper,  all  positive  angles  are  defined 
counterclockwise.  When  choosing  the  elbow  up  configura¬ 
tion,  the  angle  q2  is  clockwise,  so  the  sign  of  q2  is  negative. 
Based  on  the  geometry,  we  can  get 


Xn  —  ^  ]Lj cos  I  ^  'qj  J , 

j= 1  \i= 1  / 


Yn  =  EL?Sin(&)’ 

3=1  \i= 1  / 

n 

(frn  =  ^  (1) 


2.2  Inverse  Kinematics 
2.2.1  Two-Link  Thumb 

As  shown  in  Figure  2,  thumb  is  assumed  as  the  two-link 
(n  =  2)  finger  and  the  other  four  fingers,  such  as  index 
finger,  middle  finger,  ring  finger,  and  little  finger,  are  con¬ 
sidered  as  three-link  fingers.  Hence,  the  angular  positions, 
angular  velocities,  and  angular  accelerations  of  each  finger 
can  be  deduced  as  follows.  According  to  equation  (1),  the 


Figure  2.  Schematic  Diagram  of  Thumb  and  Index  Finger 


fingertip  coordinate  {Xt,Yt)  of  thumb  it)  can  be  described 


2.2.2  Three-Link  Index  Finger 

Figure  3  shows  the  illustration  of  three-link  (n  =  3)  index 


finger,  d  is  the  distance  between  the  global  (G)  frame  and 
index  finger  (i)  local  frame  base  (zero);  L\ ,  L2 ,  and  L\ 
are  the  lengths  of  the  links  1,  2  and  3  of  the  index  finger 
(i),  respectively;  q\ ,  q2 ,  and  q\  are  the  angles  of  the  joints 
1,  2,  and  3  of  the  index  finger.  Using  Denavit- Hart enb erg 
(DH)  [11-15]  transformation  matrices,  the  fingertip  (end- 
effector)  coordinate  P2  (X1,  Yl)  and  the  orientation  q of 
the  index  finger  (i)  can  be  written  as 

X1  =  d  +  L\  cos (q\)  +  L2  cos (q\  +  q2)  +  L\  cos (02), 
Yl  =  L\  sin(^)  +  L\  sin(^  +  q\)  +  L\  sin(^), 

(fr1  =  q\  +  ql2  +  q\.  (6) 
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The  local  coordinate  P2  (X2,  Y2)  of  the  joint  2  can  be  ob¬ 
tained  as 


Xl2  =  X{  -  L\  cos(^)  -  d , 

Yi  =  Yi  —  L|  sin(0*).  (7) 


Choosing  the  elbow  down  configuration,  the  angles  q2  and 
q\  can  be  obtained  as 


q2  =  cos 


_i  (X?  +  Y? -L\2 -L\2' 
2  L\L\ 


q\  =  tan 


Yl 


tan 


L\  sin(?2) 


Xi  j  —  ^ £*  +  I,*  cos(qi)J  ' 

Then  we  can  get  the  angle  q\  of  the  joint  3  as 

93  =  <r  -  d  -  «2- 


(8) 


(9) 


2.3  Differential  Kinematics 

The  linear  velocity  °Pn  and  angular  velocity  °u;n  of  the 
origin  of  the  end-effector  frame  are  linear  relations  to  the 
joint  angular  velocities  q  [12].  The  linear  relation  repre¬ 
sents  the  differential  kinematics  equation  and  can  be  ex¬ 
pressed  as 


°Vn  =  J  (q)  q.  (10) 

Here,  °Vn  =  [°Pn  °u;n]'  is  the  (6  x  1)  end-effector  veloc¬ 
ity  vector.  J(q)  =  [Jp(q)  Jo(q)]/  is  the  (6  x  n)  geomet¬ 
ric  Jacobian  matrix  of  a  serial  n-link  revolute-joint  planar 
manipulator;  both  position  Jacobian  Jp  (q)  and  orientation 
Jacobian  Jo(q)  are  the  (3  x  n)  matrices  which  contribute 
the  joint  angular  velocities  q  to  the  linear  velocity  °Pn  and 
angular  velocity  °u;n  of  the  end-effector,  respectively.  The 
entry  Jij  (i,j  G  N)  of  the  geometric  Jacobian  J  (q)  in  (10) 
can  be  rewritten  as 


Jij  —  \ 


-J2LjSm  (  J 

j= 1  \k=l  J 

J2ljc°s 

3= 1  U=1  / 

0 

1 


if  *  =  1 ,3  e  [1,6]; 


if  *  =  2  ,j  e  [1,6]; 

ifi  e[3,5],ie[l,6]; 
if*  =  6,j  e  [1,6]. 


Then,  the  angular  velocities  q  can  be  written  as 


q  =  J_1(q)  °V„ 


(ID 


Similarly,  the  angular  accelerations  q  can  be  obtained  as 

q  =  J-1(q)  (°A„  -  J(q)q)  .  (12) 

Here,  °An  =  [°Pn  °an]'  is  the  (6  x  1)  end-effector  accel¬ 
eration  vector,  including  linear  acceleration  °Pn  and  angu¬ 
lar  acceleration  °an.  The  differential  Jacobian  J  (q)  can  be 


also  written  as 


n  /  j 

-E  Lj  cos  E«*; 
3=  1  \fc=i 

-ELJsin  ( E^ ) 

3=  1  \k=i  j 

0 


if  j  =  1J  e  [1,6]; 

if  j  =  1  j  e  [1,6]; 
if  i  e  [3, 6],  j  €  [1,6]. 


2.4  Trajectory  Planning 


The  trajectory  planning  using  cubic  polynomial  for  finger¬ 
tip  position  control  was  discussed  in  our  previous  work 
[2, 3, 6, 10, 12]  for  a  two-fingered  (thumb  and  index  finger) 
smart  prosthetic  hand.  For  the  three-link  index  finger,  we 
present  the  same  technique  for  fingertip  orientation  control. 
A  time  history  of  desired  (d)  fingertip  orientation  (0)  and 
its  differentiation  (cj)  and  <j>)  is  given  as 


m 

—  CJq  +  Ld\t  +  CJ2t2  +  CJ3 13, 

(13) 

m 

=  Ld\  +  2id2t  +  3(x)3  t2, 

(14) 

m 

=  2cj2  +  6c U3t, 

(15) 

where  cjo-^3  are  undetermined  constants  and  the  super¬ 
script  j  indicates  the  index  of  each  finger  (j  =  i9  m,  r  and 
/).  The  relations  (13)  and  (14)  need  to  satisfy  the  constraint 
conditions  at  initial  time  to  and  final  time  t/.  This  can  be 
written  as 


T  =  $. 

Here,  the  matrices  T ,  F2,  and  are 


n 

$ 


=  ^0 


to 

1 

tf 

1 

Ux 

K 


t2 

l0 


t 3 
l0 


2 t0  3tg 


t2 

lf 

2tf 

*1 


% 

LJS 


(16) 


(17) 

(18) 
(19) 


Therefore,  the  4  unknown  constants,  cjo-^3,  can  be  com¬ 
puted  by  12  =  T-1<3>. 


2.5  Dynamics  of  Hand 


The  dynamic  equations  of  hand  motion  are  derived  via  La- 
grangian  approach  using  kinetic  energy  and  potential  en¬ 
ergy  as  [8, 11, 12, 14, 15] 


d_  (d_C\  _dC_ 
dt\d  qy  dq  T 


(20) 


where  C  is  the  Lagrangian;  q  and  q  represent  the  angular 
velocity  and  angle  vectors  of  joints,  respectively;  r  is  the 
given  torque  vector  at  joints.  The  Lagrangian  C  can  be 
expressed  as 


C  =  T-V, , 


(21) 
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where  T  and  V  denote  kinetic  and  potential  energies,  re¬ 
spectively.  Substituting  (21)  into  (20),  dynamic  equations 
can  be  obtained  as  below. 

M(q)q+ C(q,q)  +  G(q)  =  r,  (22) 

where  M(q)  is  the  inertia  matrix;  C(q,  q)  is  the  Corio¬ 
lis/centripetal  vector  and  G(q)  is  the  gravity  vector.  (22) 
can  be  also  written  as 

M(q)q  +  N(q,  q)  =  r,  (23) 


from  which  the  control  function  can  be  defined  as 

u(t)  =  q d(t)  +  M_1  (q(t))  [N(q(f),  q(f))  -  r(t)]  .  (31) 

This  is  often  called  th q  feedback  linearization  control  law, 
which  can  also  be  inverted  to  express  it  as 

r(t)  =  M(q(t))  [q d(t)  -  u (£))  +  N(q(t),  q (t)]  .  (32) 

Using  the  relations  (29)  and  (31),  and  defining  state  vector 
x(t)  =  [e'(t)  e' (£)]',  the  tracking  error  dynamics  can  be 
written  as 


where  N(q,  q)  =  C(q,  q)  +  G(q)  represents  nonlinear 
terms.  The  dynamic  relations  for  the  two-link  thumb  and 
the  three-link  index  finger  are  quite  lengthy  and  omitted 
here  due  to  lack  of  space  [12, 16, 17]. 


3  Control  Techniques 

3.1  Feedback  Linearization 


The  nonlinear  dynamics  represented  by  (23)  is  to  be  con¬ 
verted  into  a  linear  state-variable  system  using  feedback 
linearization  technique  [8].  Alternative  state- space  equa¬ 
tions  of  the  dynamics  can  be  obtained  by  defining  the  posi¬ 
tion/velocity  state  x(t)  of  the  joints  as 

x(t)  =  [q'(t)  q '(*)]'.  (24) 


Let  us  repeat  the  dynamical  model  and  rewrite  (23)  as 
|q (t)  =  -M-Hq (t))  [N(q(t),q(t))  -  r(t)] .  (25) 

Thus,  from  (24)  and  (25),  we  can  derive  a  linear  system  in 
Brunov  sky  canonical  form  as 


x(t)  = 


0  I 
0  0 


x(t)  + 


(26) 


with  its  control  input  vector  given  by 


U (t)  =  -M  1(q(i))  [N(q(f),q(f))  -  r(t)] .  (27) 


x(t)  = 


0  I 
0  0 


x(t)  + 


0 

I 


u  (t). 


(33) 


Note  that  this  is  in  the  form  of  a  linear  system  such  as 


x(t)  =  A  x(t)  +  Bu(f).  (34) 


3.2  Finite-Time  Linear  Quadratic  Optimal  Control 

Figure  4  shows  the  block  diagram  of  finite-time  linear 


Linear  System 


Figure  4.  Block  Diagram  of  Hybrid  Optimal  Controller  for 
Prosthetic  Hand 


Let  us  suppose  the  prosthetic  hand  is  required  to  track  the 
desired  trajectory  q d(t)  described  under  path  generation  or 
tracking.  Then,  the  tracking  error  e(t)  is  defined  as 

e(£)  =  q d(t)  -  q (t).  (28) 

Here,  q  d(t)  is  the  desired  angle  vector  of  joints  and  can  be 
obtained  by  trajectory  planning  [2, 3, 6, 10, 12];  q (t)  is  the 
actual  angle  vector  of  joints.  Differentiating  (28)  twice,  to 
get 

e(t)  =  q d{t)  -  q (t),  e(t)  =  qd(t)  -  q(t).  (29) 

Substituting  (25)  into  (29)  yields 

e(t)  =  qd(t)  +  M-1(q(£))  [N(q(t),  q(t))  -  r(t)]  (30) 


quadratic  optimal  controller  for  prosthetic  hand.  For  the 
linear  system  (34),  we  can  formulate  the  well-known  finite¬ 
time  linear  quadratic  optimal  control  problem  by  defining 
a  performance  index  J  [4]  such  as 

J  =  ^x'(tf)F(tf)x(tf) 

+  -  f  [x'(t)Q(t)x(t)  +  ux(t)R(t)u(t)]  dt  (35) 
2  Jt0 

where  the  terminal  cost  matrix  F  (tf)  and  the  error 
weighted  matrix  Q  (t)  are  positive  semidefinite  matrices, 
respectively;  the  control  weighted  matrix  R (t)  is  a  positive 
definite  matrix.  The  optimal  control  u*  (t)  is  given  by 

u*  (t)  =  — R_1(t)B/P(t)x*  (t)  =  — K(t)x*  (t).  (36) 
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Here,  K (t)  =  R_1(t)B/P(t)  is  called  Kalman  gain  and 
P  (t),  the  symmetric  positive  definite  matrix  (for  all  t  G 
[to,tf])9  is  the  solution  of  the  matrix  differential  Riccati 
equation  (DRE) 

P(t)  =  — P(t)A  -  A'P(t)  -  Q (t)  +  P (t)BR^ (t) B'P (t)  (37) 

satisfying  the  final  condition 

P  (t  =  tf )  =  F  (tf).  (38) 

The  optimal  state  x*  is  the  solution  of 

x*(i)  =  [A  -  BR-'^B'P (i)]  x*(i).  (39) 

Therefore,  the  required  torque  r*(f)  can  be  calculated  by 
the  optimal  control  u *(t). 

T*(t)  =  M(q(f))(qd(f)  -  u *(t))  +  N(q(f),  q(f)).  (40) 

3.3  The  Stability  of  Finite-Time  Optimal  Control 

Our  previous  work  [5,  6,  12]  shows  optimal  control  can 
avoid  overshooting  and  get  better  results  than  PID  control, 
but  the  regular  optimal  control  takes  time  to  be  convergent. 
To  study  the  convergent  stability  of  the  optimal  control,  we 
modify  the  performance  index  J  [4]  as 

J  =  ^X(U)F(U)X(U)  + 

-  f  e2at  [x'(t)Q(t)x(t)  +  ux(t)R(t)u(t)]  dt,  (41) 
2  7t0 

where  a  is  a  positive  parameter.  To  find  the  optimal  con¬ 
trol  which  minimizes  the  performance  index  (41)  under  the 
dynamical  constraint  (34).  This  problem  can  be  solved  by 
changing  the  previous  system,  so  the  following  transforma¬ 
tions  can  be  developed  as 

x(£)  =  eatx(£);  u  (t)  =  eatu  (t).  (42) 

Then,  using  the  transformations  (42),  it  is  easy  to  see  that 
the  modified  system  becomes 

£(t)  =  ^{ea‘x(f)}  =  aea‘x(f)  +  eatx(f) 

=  ak(t)  +  eat[Ax(f)  +  Bu(f)] 
x(£)  =  (A  +  oT)x(t)  +  Bu(t).  (43) 

Considering  the  minimization  of  the  modified  system 
(43)  and  the  performance  index  (41),  we  see  that  the  opti¬ 
mal  control  [4]  is  given  by 

u  *(t)  =  — R_1B'Px*(t)  =  — Kx*(t),  (44) 

where  K  =  R_1B'P  and  the  matrix  P  is  the  positive  def¬ 
inite,  symmetric  solution  of  the  algebraic  Riccati  equation 

P(A  +  al)  +  (A7  +  aI)P  -  PBR_1B'P  +  Q  =  0.  (45) 

Using  the  optimal  control  (44)  in  the  modified  system  (43), 
we  get  the  optimal  closed-loop  system  as 

i*(t)  =  (A  +  aT  —  BR-1B'P)x*(f).  (46) 


Hence,  applying  the  transformations  (43)  in  the  modified 
system  (44),  the  optimal  control  of  the  original  system  (34) 
and  the  associated  performance  measure  (41)  is  given  by 

u*(f)  =  e~atu  *(t)  =  — e_atR_1B'Pe“tx*(f) 

=  — R_1B,Px*  (t)  =  — Kx*(f).  (47) 

We  see  that  the  closed-loop  optimal  control  system  (46)  has 
eigenvalues  with  real  parts  less  than  —a.  In  other  words, 
the  state  x*(£)  approaches  zero  at  least  as  fast  as  e~at. 


4  Simulation  Results  and  Discussion 

Referring  to  inverse  kinematics,  Figure  5  exhibits  that  the 


Figure  5.  The  Fingertip  Workspace  of  Thumb  and  Index 
Finger  and  a  Square  Object 


fingertip  workspace  of  the  thumb  and  index  finger.  We 
present  simulations  with  the  stability  of  optimal  controller 
for  the  two-link  thumb  and  three-link  index  finger  of  a 
smart  prosthetic  hand.  All  used  parameters  are  the  same 
to  our  previous  work  [6].  The  initial  and  final  orienta¬ 
tion  fio  and  fif  are  75  and  160  degrees,  respectively.  Fig¬ 
ure  6  displays  the  tracking  errors  and  desired/actual  angles 
of  thumb  for  the  smart  prosthetic  hand  by  using  the  pre¬ 
sented  optimal  controller  with  the  stability  {a  =  0,  0.1,  and 
1).  Figure  7  to  Figure  9  are  the  tracking  errors  and  de¬ 
sired/actual  angles  of  index  finger  for  the  smart  prosthetic 
hand  (a  =  0,  1,  and  10).  To  compare  with  the  original  sys¬ 
tem  (a  =  0),  these  results  demonstrate  that  the  presented 
optimal  controller  behaves  fast  convergent  and  accurate  as 
the  stability  a  increases.  For  the  three-link  index  finger, 
we  design  the  orientation  to  calculate  the  angular  posi¬ 
tions  via  inverse  kinematics.  To  compare  with  our  previous 
works  [5,6, 10, 12]  by  using  adaptive  neuro-fuzzy  inference 
system  (ANFIS)  and  genetic  algorithm  (GA),  the  simula¬ 
tions  show  that  the  presented  method  is  not  time  consum¬ 
ing. 
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Figure  6.  Tracking  Errors  (left)  and  Joint  Angles  (right)  of 
Thumb  for  Optimal  Control  with  the  Stability  (a  =  0,  0.1, 
and  1) 


Figure  7.  Tracking  Errors  (left)  and  Joint  1  Angles  q\ 
(right)  of  Index  Finger  for  Optimal  Control  with  the  Sta¬ 
bility  (a  =  0,  1,  and  10) 


5  Conclusion  and  Future  Work 

This  paper  developed  the  differential  kinematics  for  a  serial 
n-link  revolute-joint  planar  prosthetic  hand  to  compute  the 
angular  velocities  and  angular  accelerations  of  each  finger. 
Then,  the  cubic  polynomial  function  was  designed  to  track 
the  orientation  f  in  order  to  calculate  inverse  kinematics 
and  differential  kinematics.  To  make  the  optimal  controller 
fast  acting,  the  performance  index  J  included  an  exponen¬ 
tial  term.  The  simulation  results  demonstrated  a  30-fold 
increase  in  fast  response  compared  to  ANFIS  or  GA  based 
trajectory  planning  [6].  Work  is  underway  to  extend  this 
methodology  to  five-fingered,  three-dimensional  prosthetic 
hand. 
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Abstract :  A  hybrid  of  soft  control  technique  of  adaptive  neuro-fuzzy  inference  system  (ANFIS)  and  genetic  algo¬ 
rithm  (GA)  and  hard  control  technique  of  proportional-integral-derivative  (PID)  for  a  five-fingered,  smart  prosthetic 
hand  is  presented.  The  ANFIS  is  used  for  inverse  kinematics  and  GA  is  used  for  tuning  the  PID  parameters  with 
the  objective  of  minimizing  the  error  squared  between  desired  and  actual  angles  of  the  links  of  the  fingers  of  the 
prosthetic  hand.  Simulation  results  for  all  the  five  fingers  with  GA-tuned  PID  controller  exhibit  superior  perfor¬ 
mance  compared  to  the  PID  control  without  GA. 

Key-Words:  Prosthetic  Hand,  PID  Control,  Genetic  Algorithm,  Adaptive  Neuro-Fuzzy  Inference  System,  Hybrid 
Control 


1  Introduction 

Hard  control  (HC)  methodologies  are  used  at  lower 
levels  for  accuracy,  precision,  stability  and  robust¬ 
ness.  HC  comprises  proportional-derivative  (PD)  con¬ 
trol  [1],  proportional-integral-derivative  (PID)  control 
[2,  3],  optimal  control  [3-6],  adaptive  control  [7-9] 
etc.  with  specific  applications  to  prosthetic  devices. 
However,  our  previous  works  [1-3,  10]  for  a  smart 
prosthetic  hand  showed  that  PID  controller  resulted  in 
overshooting  and  oscillation  because  the  system  dy¬ 
namics  are  sensitive  to  the  rigidity  of  the  target  ob¬ 
ject  and  the  used  gain  parameters  of  PD  or  PID  con¬ 
troller  [11]. 

Soft  computing  (SC)  or  computational  intelli¬ 
gence  (Cl)  is  an  emerging  field  based  on  synergy  and 
seamless  integration  of  neural  networks  (NN),  fuzzy 
logic  (FL)  and  optimization  methods,  such  as  genetic 
algorithms  (GA),  particle  swarm  (PS)  [1, 12, 13],  tabu 
search  (TS)  [13]  and  so  on.  The  methodology  based 
on  SC  can  be  used  at  upper  levels  of  the  overall  mis¬ 
sion  whereas  the  HC  can  be  used  at  lower  levels  for 
accuracy,  precision,  stability  and  robustness.  Hence, 
we  propose  the  GA-based  PID  controller  to  solve 
problems  that  cannot  be  solved  satisfactorily  by  us¬ 
ing  either  HC  or  SC  methodology  alone  with  specific 
applications  to  prosthetics. 

In  this  paper,  we  first  consider  briefly  trajec¬ 


tory  planning  and  kinematics  problems.  Then,  adap¬ 
tive  neuro-fuzzy  inference  system  (ANFIS)  is  used  to 
solve  inverse  kinematics  problem  for  three-link  fin¬ 
gers  (index,  middle,  ring,  and  little).  Next,  the  dy¬ 
namics  of  the  hand  is  derived  and  feedback  lineariza¬ 
tion  technique  is  used  to  obtain  linear  tracking  error 
dynamics.  Then  we  propose  the  GA-based  PID  con¬ 
trol,  which  uses  GA  to  tune  all  PID  parameters  by 
minimizing  the  tracking  errors,  for  the  five-fingered 
prosthetic  hand.  The  resulting  overall  hybrid  system 
incorporating  both  soft  and  hard  control  techniques  is 
simulated  with  practical  data  for  the  hand  and  found 
to  be  superior  to  that  using  PID  alone.  We  finally  pro¬ 
vide  conclusions  and  future  work. 


2  Modeling 

2.1  Trajectory  Planning  and  Kinematics 

The  trajectory  planning  using  cubic  polynomial  was 
discussed  in  our  previous  work  [1,  2,  5,  8,  9,  14]  for 
a  two-fingered  (thumb  and  index  finger)  smart  pros¬ 
thetic  hand.  The  inverse  and  differential  kinematics 
of  two-link  thumb  and  three-link  fingers  were  dis¬ 
cussed  in  our  previous  publications  [1,2,5,8,9,14]  for 
a  two-fingered  (thumb  and  index  finger)  smart  pros¬ 
thetic  hand. 
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For  five  fingers  shown  in  Figure  1,  XG ,  YG ,  and 


Figure  1 :  The  Relationship  between  Global  Coordi¬ 
nate  and  Local  Coordinates 

ZG  are  the  three  axes  of  the  global  coordinate.  The 
local  coordinate  xt-yt-zt  of  thumb  can  be  reached  by 
rotating  through  angles  a  and  f3  to  XG  and  Y G  of  the 
global  coordinate,  subsequently.  The  local  coordinate 
xl-yl-zL  of  index  finger  can  be  obtained  by  rotating 
through  angle  a  to  XG  and  then  translating  the  vector 
d1  of  the  global  coordinate;  similarly,  the  local  coor¬ 
dinate  xi-yi-z^  of  middle  finger  (j  —  m),  ring  finger 
(j  =  r),  and  little  finger  ( j  —  l )  can  be  obtained  by  ro¬ 
tating  through  angle  a  to  XG  and  then  translating  the 
vector  cF  (j  —  m,r  and  l)  of  the  global  coordinate. 

2.2  Adaptive  Neuro-Fuzzy  Inference  System 
(ANFIS) 

The  inverse  kinematics  problems  can  be  solved  by  us¬ 
ing  adaptive  neuro-fuzzy  inference  system  (ANFIS) 
method  [15]  where  the  input  of  fuzzy-neuro  system  is 
the  Cartesian  space  and  the  output  is  the  joint  space. 
ANFIS  tunes  the  membership  function  and  identifies 
the  coefficients  by  the  backpropagation  gradient  de¬ 
scent  and  least-squares  methods,  respectively.  Fig¬ 
ure  2  (a)  shows  a  two  input  first-order  Sugeno  fuzzy 
model  with  two  rules  and  Figure  2  (b)  depicts  the 
equivalent  ANFIS  structure  for  all  the  computation 
below.  Sugeno-type  fuzzy  system  has  the  following 
Rule  Base  [15]. 

If  xis  Ai  and  y  is  B\ ,  then  fi  -p\x  +  q\y  +  rq. 

If  x  is  A2  and  y  is  £>2,  then  /2  =  P2X  +  q2y  +  r2. 

Here,  x  and  y  are  inputs  to  constitute  the  premise  pa¬ 
rameters  Ai,  A2,  B 1,  and  B2  (Layer  1  in  Figure  2 
(b)).  pi,  qi ,  and  r{  ( i  =  1,2)  are  the  consequent  param¬ 
eters.  We  evaluate  the  rules  by  choosing  product  ]j 
for  T-Norm  (Layers  2  and  3)  which  results  in 

Wi  =  VAiix)  i  =  1,2.  (1) 


f=pxx  +  qy  +  rx 

f=  Wif  + 

J  Wl+w2 
=  wjx+  wj2 

t 

f2=p2x  +  q2y  +  r2 


(a)  A  Two  Input  First-Order  Sugeno  Fuzzy  Model  with  Two  Rules 

Layer  1  Layer  2  Layer  3  Layer  4  Layer  5 


Figure  2:  ANFIS  Architecture:  (a)  A  Two  Input 
First-Order  Sugeno  Fuzzy  Model  with  Two  Rules  (b) 
Equivalent  ANFIS  Structure  [15] 


Here,  PAi(x)  and  PBi(y)  are  designed  fuzzy  mem¬ 
bership  functions.  Now  after  leaving  the  arguments 
(Layer  4),  we  get  the  output  f(x,y)  by  Rule  Conse¬ 
quences. 


wi(x,y)fi(x,y)  +  w2(x,y)f2(x,y) 

f{x,y)  = - 7 - r— - 7 - t - •  (2) 

w1{x,y)  +  w2{x,y) 

/(Layer  5)  can  be  written  as 

Wifi  +  W2h  -  t  ,  _  t 

J  = - ; - =  wifi  +  w2h ,  (3) 

w  1  +  W2 

where 

wi  -  w2  ... 

w  1  =  - ; - ,  w2  =  - ; - •  (4) 

W  i  +W2  Wi  +  W2 

2.3  Dynamics  of  the  Prosthetic  Hand 

The  dynamic  equations  of  hand  motion  are  derived  via 
Lagrangian  approach  using  kinetic  energy  and  poten¬ 
tial  energy  as  [7, 14, 16]  and  can  be  written  as  below. 

M(q)q  +  N(q,  q)  —  r,  (5) 

where  M(q)  is  the  inertia  matrix  and  N(q,  q)  = 
C(q,  q)  +  G(q)  represents  nonlinear  terms,  including 
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Coriolis/centripetal  vector  C(q,  q)  and  gravity  vector 
G(q).  The  dynamic  relations  for  the  two-link  thumb 
and  the  remaining  three-link  fingers  are  quite  lengthy 
and  omitted  here  due  to  lack  of  space  [14]. 

3  Control  Techniques 

3.1  Feedback  Linearization 

The  nonlinear  dynamics  represented  by  (5)  is  to  be 
converted  into  a  linear  state-variable  system  using 
feedback  linearization  technique  [7].  Let  us  suppose 
the  prosthetic  hand  is  required  to  track  the  desired 
trajectory  q d(t)  described  under  path  generation  or 
tracking.  Then,  the  tracking  error  e(t)  is  defined  as 

e(t)  =  ~  (6) 

Here,  q d(t)  is  the  desired  angle  vector  of  joints  and 
can  be  obtained  by  trajectory  planning  [1, 2, 5, 8, 14]; 
q (£)  is  the  actual  angle  vector  of  joints.  Differentiat¬ 
ing  (6)  twice,  to  get 

e(t)  =  q d(t)  -  q(t)»  e(t)  =  qd(t)  -  q (t).  (7) 

Substituting  (5)  into  (7)  yields 

e(t)  =  qd(t)  +M~1(q(t))  [N(q(f),  q(f))  -  r(f)]  (8) 

from  which  the  control  function  can  be  defined  as 

u(f)  =  qd(t)  +M~1(q(t))  [N(q(t),q(t))  -  r(f)] .  (9) 


Figure  3:  Block  Diagram  of  the  Hybrid  GA-Based 
PID  Controller  for  14-DOF  Five-Fingered  Prosthetic 
Hand 

Then  we  use  GA  to  tune  all  gain  coefficients  Kp, 
Kd  and  Kd  of  PID  controller.  Figure  4  shows  the 
flowchart  of  GA  and  the  procedure  is  briefly  stated 
below. 


This  is  often  called  the  feedback  linearization  control 
law,  which  can  also  be  inverted  to  express  it  as 

r(f)  =  M(q (t))  [qa(t)  -  u(i))  +  N(q(i),q(i)] .  (10) 


Using  the  relations  (7)  and  (9),  and  defining  state  vec¬ 
tor  x(£)  =  [e'(t)  the  tracking  error  dynamics 

in  the  form  of  a  linear  system  can  be  written  as 


x(t)  = 


0  I 
0  0 


x(*)  + 


0 

I 


u  (*)• 


(11) 


3.2  GA-Based  PID  Hybrid  Control 

Figure  3  shows  the  block  diagram  of  a  hybrid  GA- 
based  PID  controller  for  the  presented  five-fingered 
prosthetic  hand  with  control  signal  as 

u (t)  =  —  Kpe(t)  —  Ki  J  e(t)dt  —  Kd e(t)  (12) 

with  the  proportional  Kp,  integral  Ki,  and  derivative 
Kd  diagonal  gain  matrices.  We  then  rewrite  (10)  as 

r(f)  =  M(q(f))[qd(f)  +  KPe(f)  +  Kj;  J  e(t)dt 
+KDe(f))  +  N(q(f),  q(f)].  (13) 


1.  Define  the  GA  parameters:  include  initial  pop¬ 
ulation,  population  at  the  end  of  the  first  gener¬ 
ation,  number  of  chromosomes  kept  for  mating, 
mutation  rate,  and  tolerance  e  so  on. 

2.  Create  a  homogeneous  population:  generate  N 
elements  (chromosomes)  and  N  is  the  initial 
population. 

3.  Evaluate  cost  (fitness)  function  of  each  chromo¬ 
some:  calculate  the  fitness  value  of  the  ith  mem¬ 
ber  in  the  population. 

4.  Select  mate  based  on  the  performance  of  each 
gene:  create  a  new  population  from  the  current 
population  based  on  the  ranking  of  the  current 
fitness  value,  e.g.  determine  which  parents  par¬ 
ticipate  in  producing  offspring  for  the  next  gen¬ 
eration. 

5.  Reproduce  the  generation  by  crossover:  use  the 
single  or  multiple  crossover  points  to  generate 
new  chromosomes  that  retain  the  good  feature 
and  discard  the  bad  feature. 
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Figure  4:  The  Flowchart  of  Genetic  Algorithm  (GA) 

6.  Mutate:  utilize  the  mutation  rate  which  can  ran¬ 
domly  mutate  the  gene  to  avoid  falling  into  the 
local  minima  area. 

7.  Repeat  steps  3  to  6  until  it  reaches  the  maximum 
number  of  iterations  or  stopping  condition  de¬ 
fined  by  e  is  satisfied. 


4  Simulation  Results  and  Discussion 

We  present  simulations  with  a  PID  controller  and  GA- 
tuned  PID  controller  for  the  14  DOFs  five-fingered 
smart  prosthetic  hand  grasping  a  rectangular  object  as 
shown  in  Figure  5.  All  parameters  of  the  smart  pros¬ 
thetic  hand  selected  for  the  simulations  are  the  same 
as  our  previous  works  [3,9].  All  initial  actual  angles 
are  zero  and  the  diagonal  coefficients,  Kp,  Ki  and 
Kd,  for  the  PID  controller  alone  are  arbitrarily  cho¬ 
sen  as  100.  From  the  derived  dynamic  and  control 
models,  after  the  parameters  (Kp,  Ki  and  Kd)  are 
determined,  the  torque  matrix  r  can  be  computed,  and 
then  the  squared-tracking  errors  ej  ( t )  of  the  joint  i  of 
the  finger  j  are  obtained.  Therefore,  the  total  error 
E(t),  which  is  a  time-dependent  function,  can  be  de¬ 
scribed  as 

E(t)  =  [tf  (14) 

Jt0 


Figure  5:  A  Five-Finger  Prosthetic  Hand  Grasping  a 
Rectangular  Object 

where  to  and  t f  are  initial  and  terminal  time,  respec¬ 
tively.  The  tuned  diagonal  parameters  (Kp,  Ki  and 
Kd)  and  the  total  error  E{t )  of  PID  controller  by 
GA  are  listed  in  Table  1 .  To  study  whether  the  tuned 

Table  1:  Parameter  Selection  of  GA-Tuned  PID  Con¬ 
troller  and  Computed  Total  Errors 


Input  Output 


Fingers 

KP 

Ki 

Kd 

m 

Case  I 

[976,956] 

[779,279] 

[170,236] 

0.3107 

Case  II 

[988,999] 

[78,848] 

[  80,109] 

0.1557 

Case  III 

[199,198] 

[127,157] 

[104,102] 

0.8100 

Index 

[794,398,960] 

[960,918,914] 

[15,59,242] 

0.0465 

Middle 

[794,398,960] 

[960,918,914] 

[15,59,242] 

0.1003 

Ring 

[794,398,960] 

[960,918,914] 

[15,59,242] 

0.0465 

Little 

[794,398,960] 

[960,918,914] 

[15,59,242] 

0.0607 

parameter  range  influences  total  tracking  errors,  we 
design  three  different  cases  with  altering  lower  and 
upper  bounds  of  tuned  parameter  ranges  for  two-link 
thumb.  Cases  I,  II,  and  III  for  the  thumb  represent 
that  the  PID  parameters  Kp,  Ki  and  Kd  are  con¬ 
stricted  in  three  different  bounded  ranges  [100,1000], 
[50,1000],  and  [100,200],  respectively.  Figure  6  and 
Figure  7  show  that  tracking  errors  and  desired/actual 
angles  of  joints  1  and  2  of  PID  and  GA-based  PID 
controllers  for  Thumb.  These  simulations  show  that 
the  large  ranges  [100,1000]  (Case  I)  and  [50,1000] 
(Case  II)  provide  better  results  than  the  PID  controller 
parameters  arbitrarily  chosen  as  100.  However,  the 
small  range  [100,200]  (Case  III)  gives  worse  result 
than  the  PID  controller  alone.  These  results  suggest 
that  the  bigger  parameter  range,  the  smaller  the  to¬ 
tal  error.  Cases  I  and  II  explain  that  GA  finds  some 
parameter  values  E  [100,1000]  and  [50,100]  escaping 


Page  331  of  531 


Figure  6:  Tracking  Errors  of  Joint  1  (left)  and  Joint 
2  (right)  of  PID  and  GA-Based  PID  Controllers  for 
Thumb 


Figure  8:  Tracking  Errors  (left)  and  Joint  Angles 
(right)  of  PID  and  GA-Based  PID  Controllers  for  In¬ 
dex  Finger 


Figure  7:  Tracking  Angles  of  Joint  1  (left)  and  Joint 
2  (right)  of  PID  and  GA-Based  PID  Controllers  for 
Thumb 


the  local  minimum  area.  Case  III  covers  the  value  100 
in  lower  bound,  but  both  total  error  and  convergent 
speed  are  even  worse  than  PID  alone,  suggesting  that 
GA  performs  better  for  a  large  range,  but  is  poor  for 
searching  on  the  boundary.  To  further  consider  the 
convergent  speed,  Case  I  gives  smaller  total  error,  but 
does  not  improve  its  convergent  speed  when  compar¬ 
ing  to  PID  control  alone.  Yet,  Case  II  gives  good  total 
error  and  convergent  speed.  Case  III  gives  poor  total 
error  and  convergent  speed.  Taken  together,  these  re¬ 
sults  imply  that  the  global  minimum  could  be  located 
in  the  ranges  [50,100]  and  [200,1000]  and  the  parame¬ 
ter  ranges  play  an  important  role  in  GA  tuning.  Based 
on  these  findings,  we  use  the  range  [50,1000]  for  the 
remaining  three-link  fingers.  Figures  8  to  1 1  show  the 
simulations  of  PID  and  GA-based  PID  controllers  for 
the  remaining  three-link  fingers. 


Figure  9:  Tracking  Errors  (left)  and  Joint  Angles 
(right)  of  PID  and  GA-Based  PID  Controllers  for 
Middle  Finger 

5  Conclusions  and  Future  Work 

A  hybrid  control  technique  combining  soft  control 
with  adaptive  neuro-fuzzy  inference  system  (ANFIS) 
and  genetic  algorithm  (GA)  and  hard  control  with 
proportional-integral-derivative  (PID)  was  presented 
for  a  five-fingered  smart  prosthetic  hand.  The  AN¬ 
FIS  is  used  for  inverse  kinematics  and  GA  is  used 
for  tuning  the  PID  parameters  with  the  objective  of 
minimizing  the  error  squared  between  desired  and  ac¬ 
tual  angles  of  the  links  of  the  fingers.  Simulation  re¬ 
sults  for  all  the  five  fingers  with  GA-tuned  PID  con¬ 
troller  showed  superior  performance  compared  to  the 
PID  control  alone.  A  real-time  implementation  of 
this  technique  on  a  prototype  of  a  prosthetic  hand  is 
planned  for  future  work. 
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Figure  11:  Tracking  Errors  (left)  and  Joint  Angles 
(right)  of  PID  and  GA-Based  PID  Controllers  for  Lit¬ 
tle  Finger 
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TABLE  I 

Parameter  Selection  of  the  Smart  Hand 


Parameters 

Values 

Thumb 

Time  (to,tf)* 

Desired  Initial  Position 

Desired  Final  Position  ( Xj,Yp ** 

Desired  Initial  Velocity  (Xq,Yq)* 

Desired  Final  Velocity  (Xj,Yp* 

Length  (L\,L\) 

Mass  (jn\,rn\) 

0,  20  (sec) 

0.035,  0.060  (m) 

0.0495,  0.060  (m) 

0,  0  (m/s) 

0,  0  (m/s) 

0.040,  0.040  (m) 

0.043,  0.031  (kg) 

Index  Finger 

Desired  Initial  Position  (X^YJ)** 

Desired  Final  Position  (X*  ,Y*)** 

Length  (L*,L*  ,L*) 

Mass  (m\ ,m\) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 
0.045,  0.025,  0.017  (kg) 

Middle  Finger 

Desired  Initial  Position  (X"\Y0m)** 
Desired  Final  Position  (X"\Y/T* 

Length  (L™,L™,L™) 

Mass  (jn™ ,m™) 

0.065,  0.080  (m) 

0.005,  0.060  (m) 

0.044,  0.044,  0.033  (m) 
0.050,  0.028,  0.017  (kg) 

Ring  Finger 

Desired  Initial  Position  (XJ,Y0r)** 

Desired  Final  Position  (. Xrf,Yj )** 

Length  ( L\,Lr2,Lr3 ) 

Mass  (m\ ,m2,m3) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 
0.041,  0.023,  0.014  (kg) 

Little  Finger 

Desired  Initial  Position  (. Xl0,Y{ p** 

Desired  Final  Position  (Xlf,Yj)** 

Length  (L[,Ll2,Ll3) 

Mass  (m\,ml2,ml3) 

0.055,  0.080  (m) 

0.020,  0.060  (m) 

0.036,  0.036,  0.027  (m) 
0.041,  0.023,  0.014  (kg) 

*A11  fingers  use  same  parameters 
**A11  parameters  are  in  local  coordinates 

TABLE  II 

Parameter  Selection  of  the  Conversion  from  Global  to 
Local  Coordinates 


Parameters 

Values 

Rotating  a 

90  (deg) 

Rotating  (3 

45  (deg) 

Translating  d1 

(0.035,  0,  0)  (m) 

Translating  dm 

(0.040,  0,  -0.020)  (m) 

Translating  dr 

(0.035,  0,  -0.040)  (m) 

Translating  d1 

(0.025,  0,  -0.060)  (m) 

TABLE  III 

GA-Tuned  Parameter  Selection  of  PID  Control  and 
Evaluated  Total  Error 


Input  Output 


Fingers 

KP 

Ki 

Kd 

m 

Case  I 

[976,956] 

[779,279] 

[170,236] 

0.3107 

Case  II 

[988,999] 

[  78,848] 

[  80,109] 

0.1557 

Case  III 

[199,198] 

[127,157] 

[104,102] 

0.8100 

Index 

[794,398,960] 

[960,918,914] 

[  15,59,242] 

0.0465 

Middle 

[794,398,960] 

[960,918,914] 

[  15,59,242] 

0.1003 

Ring 

[794,398,960] 

[960,918,914] 

[  15,59,242] 

0.0465 

Little 

[794,398,960] 

[960,918,914] 

[  15,59,242] 

0.0607 
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Abstract — A  hybrid  of  soft  control  technique  of  adaptive  neuro- 
fuzzy  inference  system  (ANFIS)  and  fuzzy  logic  (FL)  and  hard 
control  technique  of  proportional-derivative  (PD)  for  a  five¬ 
fingered,  smart  prosthetic  hand  is  presented.  The  ANFIS  is 
used  for  inverse  kinematics  and  FL  is  used  for  tuning  the  PD 
parameters  with  two  input  layers  (error  and  error  change)  using 
7  triangular  membership  functions  and  49  fuzzy  logic  rules. 
Simulation  results  with  FL-tuned  PD  controller  exhibit  superior 
performance  compared  to  the  PD  and  PID  control  alone. 

Index  Terms — Prosthetic  Hand,  Fuzzy  Logic,  PD  Control, 
Adaptive  Neuro-Fuzzy  Inference  System,  Hybrid  Control 

I.  Introduction 

During  the  recent  wars  in  Afghanistan  and  Iraq,  “at  least 
251,102  people  have  been  killed  and  532,715  people  have 
been  seriously  wounded”  [1].  Further,  in  the  United  States, 
the  Amputee  Coalition  of  America  (ACA)  [2]  reports  that 
there  are  approximately  1.9  million  people  living  with  limb 
loss,  due  to  combat  operations  (such  as  conflicts,  wars  etc.), 
and  non-combat  operations  such  as  accidents,  or  birth  defects. 
According  to  a  study  of  the  1996  National  Health  Interview 
Survey  (NHIS)  published  by  Vital  and  Health  Statistics  [3], 
it  is  estimated  that  one  out  of  every  200  people  in  the  U.S. 
has  had  an  amputation.  In  other  words,  about  one  in  every 
2,000  new  born  babies  will  have  limb  deficiency  and  over 
3,000  people  lose  a  limb  every  week  in  America.  By  the  year 
2050,  the  projected  number  of  Americans  living  with  limb 
amputation  will  become  3.6  million  [4].  In  the  past  several 
decades,  prosthetic  hands  have  been  developed  by  various 
researchers  in  the  field  [5]— [9],  but  reproducing  the  human 
hand  in  all  its  various  functions,  appearance,  visibility,  and 
weight  is  still  a  challenging  task  [5]. 

Hard  computing/control  (HC)  techniques  are  used  at 
lower  levels  for  accuracy,  precision,  stability  and  robust¬ 
ness.  HC  comprises  proportional-derivative  (PD)  control  [10], 
proportional-integral-derivative  (PID)  control  [11],  [12],  op¬ 
timal  control  [12]— [15],  adaptive  control  [16]— [19]  etc.  with 
specific  applications  to  prosthetic  devices.  However,  our  previ¬ 
ous  works  [10]-[12],  [20]  for  a  smart  prosthetic  hand  showed 
that  PID  controller  resulted  in  overshooting  and  oscillation. 


Soft  computing  (SC)  or  computational  intelligence  (Cl)  [21]  is 
an  emerging  field  based  on  synergy  and  seamless  integration 
of  neural  networks  (NN),  fuzzy  logic  (FL)  and  optimization 
methods,  such  as  genetic  algorithms  (GA),  particle  swarm 
(PS)  [10],  [22],  [23],  tabu  search  (TS)  [23]  and  so  on.  The 
methodology  based  on  SC  can  be  used,  in  particular  with  FL,  at 
upper  levels  of  the  overall  mission  where  human  involvement 
and  decision  making  is  of  primary  importance,  whereas  the  HC 
can  be  used  at  lower  levels  for  accuracy,  precision,  stability  and 
robustness.  Hence,  we  present  the  integration  of  SC  and  HC 
to  solve  problems  that  cannot  be  solved  satisfactorily  by  using 
either  HC  or  SC  methodology  alone  with  specific  applications 
to  prosthetics. 

In  this  paper,  we  first  consider  briefly  trajectory  plan¬ 
ning  and  kinematics  problems  associated  with  a  five-fingered 
prosthetic  hand.  Then,  adaptive  neuro-fuzzy  inference  system 
(ANFIS)  is  used  to  solve  inverse  kinematics  problem  for  three- 
link  fingers  (index,  middle,  ring,  and  little).  Next,  the  dynamics 
of  the  hand  is  derived  and  feedback  linearization  technique 
is  used  to  obtain  linear  tracking  error  dynamics.  Then  we 
propose  the  fuzzy  logic-based  PD  (hybrid)  controller,  which 
uses  fuzzy  logic  with  7  triangular  membership  functions  in 
each  input/output  layer  and  49  logic  rules  to  tune  PD  gain 
parameters.  The  resulting  overall  hybrid  system  incorporating 
both  soft  and  hard  control  techniques  is  simulated  with  practi¬ 
cal  data  for  the  hand  and  found  the  performance  to  be  superior 
to  PD  or  PID  alone.  Finally,  we  provide  conclusions  and  future 
work. 

II.  Modeling 
A.  Trajectory  Planning  and  Kinematics 

The  trajectory  planning  using  cubic  polynomial  was  dis¬ 
cussed  in  our  previous  work  [10],  [11],  [14],  [18],  [19],  [24] 
for  a  two-fingered  (thumb  and  index  finger)  smart  prosthetic 
hand. 

Kinematics  is  the  study  of  geometry  in  motion  and  is 
restricted  to  a  natural  geometrical  description  of  motion  by 
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the  manners,  including  positions,  orientations,  and  their  deriva¬ 
tives  (velocities  and  accelerations)  [25],  [26].  Forward  and 
inverse  kinematics  of  articulated  systems  study  the  analytical 
relationship  between  the  angular  positions  of  joints  and  the 
positions  and  orientations  of  the  fingertips.  A  desired  trajectory 
is  usually  specified  in  Cartesian  space  and  the  trajectory 
controller  is  easily  performed  in  the  joint  space.  Therefore, 
conversion  of  Cartesian  trajectory  planning  to  the  joint  space 
[26]  is  necessary.  Using  inverse  kinematics,  the  joint  angular 
positions  of  each  finger  need  to  be  obtained  from  the  known 
fingertip  positions.  Then  the  angular  velocities  and  angular 
accelerations  of  joints  can  be  obtained  from  the  linear  and 
angular  velocities  and  accelerations  of  fingertips  by  differential 
kinematics.  The  inverse  and  differential  kinematics  of  two-link 
thumb  and  three-link  fingers  were  discussed  in  our  previous 
publications  [10],  [11],  [14],  [18],  [19],  [24]  for  a  two-fingered 
(thumb  and  index  finger)  smart  prosthetic  hand. 

For  five  fingers  shown  in  Figure  1,  XG ,  YG ,  and  ZG  are 


Fig.  1 .  Relationship  between  Global  Coordinate  and  Local  Coordinates:  Local 
coordinate  xt-yt-zt  of  thumb  can  be  reached  by  rotating  through  angles  a 
and  (3  to  XG  and  YG  of  global  coordinate,  subsequently.  Local  coordinate 
x'l-y'l-z'1  of  index  finger  can  be  obtained  by  rotating  through  angle  a  to  XG 
and  then  translating  a  vector  d1  of  the  global  coordinate. 


networks  which  provides  the  best  optimization  algorithm  to 
find  parameters  in  order  to  fit  the  data.  ANFIS  includes 
premise  parameters ,  that  defines  membership  functions,  and 
consequent  parameters ,  that  defines  the  coefficients  of  each 
output  equation.  ANFIS  tunes  the  membership  function  and 
identifies  the  coefficients  by  the  backpropagation  gradient 
descent  and  least- squares  methods,  respectively.  ANFIS  uses 
a  hybrid  learning  algorithm  to  identify  parameters  of  Sugeno- 
type  fuzzy  inference  systems  for  fuzzy  modeling  procedures 
to  learn  information  about  a  data  set  in  order  to  compute  the 
membership  function  and  track  the  given  input/output  data. 
Figure  2  (a)  shows  a  two  input  first-order  Sugeno  fuzzy  model 
with  two  rules  and  Figure  2  (b)  depicts  the  equivalent  ANFIS 
structure  for  all  the  computation  below.  Sugeno-type  fuzzy 


f=pxx  +  qy  +  rx 

f=  Wl/i'  +  Wzfi 
J  Wl+W2 

=  wjx+  wj2 

t 

f2=p2x  +  q2y  +  r2 


(a)  A  Two  Input  First-Order  Sugeno  Fuzzy  Model  with  Two  Rules 

Layer  1  Layer  2  Layer  3  Layer  4  Layer  5 


(b)  Equivalent  ANFIS  Structures 


the  three  axes  of  global  coordinate.  Local  coordinate  xt-yt-zt 
of  thumb  can  be  reached  by  rotating  through  angles  a  and  f3 
to  XG  and  YG  of  the  global  coordinate,  subsequently.  Local 
coordinate  xl-yl-zl  of  index  finger  can  be  obtained  by  rotating 
through  angle  a  to  XG  and  then  translating  a  vector  d1  of  the 
global  coordinate;  similarly,  the  local  coordinate  x^-y^-z^  of 
middle  finger  (j  =  m),  ring  finger  (j  =  r),  and  little  finger 
(j  =  l )  can  be  obtained  by  rotating  through  angle  a  to  XG 
and  then  translating  the  vector  cF  (j  =  m,  r  and  l)  of  the 
global  coordinate. 

B.  Adaptive  N euro -Fuzzy  Inference  System  (ANFIS) 

The  inverse  kinematics  problems  can  be  solved  by  us¬ 
ing  adaptive  neuro-fuzzy  inference  system  (ANFIS)  method 
[27]  where  the  input  of  fuzzy-neuro  system  is  the  Cartesian 
space  and  the  output  is  the  joint  space.  ANFIS  is  a  fuzzy 
inference  system  implemented  in  the  framework  of  adaptive 


Fig.  2.  ANFIS  Architecture:  (a)  A  Two  Input  First-Order  Sugeno  Fuzzy 
Model  with  Two  Rules  (b)  Equivalent  ANFIS  Structure  [27].  The  inverse 
kinematics  problems  are  solved  by  using  adaptive  neuro-fuzzy  inference  sys¬ 
tem  (ANFIS)  method  where  the  input  of  fuzzy-neuro  system  is  the  Cartesian 
space  and  the  output  is  the  joint  space.  ANFIS  is  a  fuzzy  inference  system 
implemented  in  the  framework  of  adaptive  networks  which  provides  the  best 
optimization  algorithm  to  find  parameters  in  order  to  fit  the  data. 

system  has  the  following  Rule  Base  [27]-[31]. 

If  v  is  A i  and  y  is  then  /i  =  p\x  +  qiy  +  r\. 

If  x  is  A2  and  y  is  B 2,  then  f2  =  P2%  +  q^V  +  r2. 

Here,  x  and  y  are  inputs  to  constitute  the  premise  parameters 
A1,  A 2,  B 1,  and  B2  (Layer  1  in  Figure  2  (b)).  pi ,  qi ,  and  r* 
(i  =  1,2)  are  the  consequent  parameters.  We  evaluate  the  rules 
by  choosing  product  for  T-Norm  (Layers  2  and  3)  which 
results  in 

Wi  =  VAi{x)  i  =  !,2.  (1) 
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Here,  /i^.(x)  and  // bXv )  are  designed  fuzzy  membership 
functions.  Now  after  leaving  the  arguments  (Layer  4),  we  get 
the  output  f(x,y)  by  Rule  Consequences. 


fix,  y )  = 


wi{x,y)f1(x,y) +w2(x,y)f2(x,y) 


wi  (x,y)  +  w2(x,y) 

f  (Layer  5)  can  be  written  as 

wifi  +  ^2/2 


(2) 


/  = 


w  1  +  ^2 
=  wifi  +  ^2/2? 


where 


wi  = 


W2  = 


W2 


(3) 


(4) 


wi  +  ^2  w  1  +  w2 

C.  Dynamics  of  Hand 

The  dynamic  equations  of  hand  motion  are  derived  via 
Lagrangian  approach  using  kinetic  energy  and  potential  energy 
as  [17],  [24]— [26],  [32] 


d 

dt 


dC 

<9q 


dC 

<9q 


=  r, 


(5) 


where  C  is  the  Lagrangian;  q  and  q  represent  the  angular  ve¬ 
locities  and  angle  vectors  of  joints,  respectively;  r  is  the  given 
torque  vector  at  joints.  The  Lagrangian  C  can  be  expressed  as 


C  =  T-V , 


(6) 


Thus,  from  (9)  and  (10),  we  can  derive  a  linear  system  in 
Brunovsky  canonical  form  as 


±{t)  = 


0  1 
0  0 


x{t)  + 


(11) 


with  its  control  input  vector  given  by 


u(t)  =  -M  [N(q(i),q(i))  -  r(t)] .  (12) 


Let  us  suppose  the  prosthetic  hand  is  required  to  track  the 
desired  trajectory  q d(t)  described  under  path  generation  or 
tracking.  Then,  the  tracking  error  e(t)  is  defined  as 

e(£)  =  q d(t)  -  q (t).  (13) 

Here,  q d(t)  is  the  desired  angle  vector  of  joints  and  can  be 
obtained  by  trajectory  planning  [10],  [11],  [14],  [18],  [24];  q (t) 
is  the  actual  angle  vector  of  joints.  Differentiating  (13)  twice, 
to  get 

e(i)  =  q d(t)  -  q (t),  e(t)  =  q d(t)  -  q (t).  (14) 

Substituting  (10)  into  (14)  yields 

e(t)  =  q d(t)  +  M_1(q(£))  [N(q(i),  q (t))  -  r(t)\  (15) 

from  which  the  control  function  can  be  defined  as 

U (*)  =  q d(t)  +  M_1(q(t))  [N(q(t),  q (t))  -  r(t)] .  (16) 

This  is  often  called  the  feedback  linearization  control  law, 
which  can  also  be  inverted  to  express  it  as 


where  T  and  V  denote  kinetic  and  potential  energies,  respec¬ 
tively.  Substituting  (6)  into  (5),  dynamic  equations  of  thumb 
can  be  obtained  as  below. 

M(q)q  + C(q,q)  +  G(q)  =  r,  (7) 

where  M(q)  is  the  inertia  matrix;  C(q,  q)  is  the  Corio¬ 
lis/centripetal  vector  and  G(q)  is  the  gravity  vector.  (7)  can 
be  also  written  as 


r(t)  =  M(q (t))  [qd(i)  -  u (t))  +  N(q (t),  q(t)]  .  (17) 


Using  the  relations  (14)  and  (16),  and  defining  state  vector 
x(£)  =  [e'(t)  the  tracking  error  dynamics  can  be 


written  as 


x(7)  = 


0  I 
0  0 


c  (t) 


u  (t). 


(18) 


Note  that  this  is  in  the  form  of  a  linear  system  such  as 


M(q)q  +  N(q,  q)  =  r ,  (8) 


x(t)  =  A  x(t)  +  Bu(t).  (19) 


where  N(q,  q)  =  C(q,  q)  +  G(q)  represents  nonlinear  terms. 
The  dynamic  relations  for  the  two-link  thumb  and  the  remain¬ 
ing  three-link  fingers  are  quite  lengthy  and  omitted  here  due 
to  lack  of  space  [24],  [33],  [34]. 

III.  Control  Techniques 
A.  Feedback  Linearization 

The  nonlinear  dynamics  represented  by  (8)  is  to  be  con¬ 
verted  into  a  linear  state-variable  system  using  feedback  lin¬ 
earization  technique  [17].  Alternative  state- space  equations  of 
the  dynamics  can  be  obtained  by  defining  the  position/velocity 
state  x(£)  of  the  joints  as 

x(i)s[q'(7)  q '(*)]'.  (9) 

Let  us  repeat  the  dynamical  model  and  rewrite  (8)  as 

|q it)  =  -M-^qC t))  [N(q(i),q (t))  -  r(t)]  .  (10) 


B.  Fuzzy  Logic-Based  PD  Hybrid  Control 

Figure  3  shows  the  block  diagram  of  the  hybrid  fuzzy  logic- 
based  PD  controller  for  the  presented  five-fingered  prosthetic 
hand  with  control  signal  as 

u(t)  =  — KP(t)e(t)  —  Kd  (t)e(t)  (20) 

with  the  proportional  Kp(£)  and  derivative  Kd  (t)  diagonal 
gain  matrices  with  time  varying.  We  then  rewrite  (17)  as 

r(t)  =  M(q(i))[qd(t)  +  KP(t)e(t)  +  K  D(t)e(t))  + 

N(q(t),q(t)].  (21) 

Then  we  use  mamdani  fuzzy  inference  system  to  tune  the 
time- varying  parameters  Kp  (t)  and  Kd  (t)  of  PD  controller. 
Figure  4  shows  the  structural  characteristics  of  proposed  fuzzy 
inference  system,  which  includes  two  inputs  (error  e  and  error 
change  e)  on  the  left  and  one  output  (Kp)  on  the  right.  Each 
input  or  output  layer  contains  seven  triangular  membership 
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Fig.  3.  Block  Diagram  of  the  Presented  Hybrid  Fuzzy  Logic-Based 
Proportional-Derivative  (PD)  Controller  for  a  Five-Fingered  Prosthetic  Hand: 
Errors  and  error  changes  are  calculated  by  actual  and  desired  angles,  which  are 
based  on  adaptive  neuro-fuzzy  inference  system  (ANFIS)  trajectory  planner. 
Then  fuzzy  logic  controller  tunes  all  parameters  of  closed-loop  PD  control  so 
that  the  required  torque  of  the  prosthetic  hand  nonlinear  system  is  computed 
by  control  input. 


ep(7l 

System  Thumb  2  inputs.  3  outputs.  -J9  nUes 


Fig.  5.  All  Membership  Functions:  Each  of  Two  Inputs  (Upper  Panel)  and 
One  Output  (Lower  Panel)  Uses  7  Triangular  Membership  Functions 


TABLE  I 

A  Complete  Fuzzy  Logic  Rule  Base 


e\e 

NL 

NM 

NS 

ZR 

PS 

PM 

PL 

NL 

ZR 

ZR 

ZR 

ZR 

VS 

S 

SM 

NM 

ZR 

ZR 

ZR 

VS 

s 

SM 

ML 

NS 
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ZR 

VS 

S 

SM 

ML 

L 

ZR 

ZR 

VS 

S 

SM 

ML 

L 

VL 

PS 

VS 

S 

SM 

ML 

L 

VL 

VL 

PM 

S 

SM 

ML 

L 

VL 

VL 

VL 

PL 

SM 

ML 

L 

VL 

VL 

VL 

VL 

N:  negative;  P:  positive;  ZR:  zero;  L:  large;  M:  medium; 
S:  small;  V:  very 


kinematics,  the  first  and  second  joint  angles  of  the  thumb 
fingertip  are  constrained  in  the  ranges  of  [0,90]  and  [-80,0] 
(degrees).  The  first,  second,  and  third  joint  angles  of  the  other 
four  fingers  are  constrained  in  the  ranges  of  [0,90],  [0,110] 
and  [0,80]  (degrees),  respectively  [35]. 

Next,  we  present  simulations  with  PD  and  PID  controllers 
and  fuzzy  inference  system  tuned  PD  controller  for  the  14 
DOFs  five-fingered  smart  prosthetic  hand.  The  parameters 
of  the  two-link  thumb/three-link  fingers  [36]  were  related  to 
desired  trajectory.  All  parameters  of  the  smart  prosthetic  hand 
selected  for  the  simulations  are  given  in  Table  II  and  the 


Fig.  4.  The  Structural  Characteristics  of  Proposed  Fuzzy  Inference  System: 
Two  inputs  (error  e  and  error  change  e)  on  the  left  and  one  output  (tuned 
KP)  on  the  right  with  49  logic  rules 


functions  as  shown  in  Figure  5  and  49  logic  rules  as  listed  in 
Table  I.  After  using  49  logic  rules,  the  output  surface  Kp 
of  fuzzy  inference  system  is  generated  as  shown  in  Figure  6. 
Similarly,  Kp  can  be  computed  by  the  same  way. 

IV.  Simulation  Results  and  Discussion 


I 
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Figure  7  shows  that  a  five-fingered  prosthetic  hand  with 
14  DOFs  is  reaching  a  rectangular  rod  in  order  to  grasp  the 
object.  When  thumb  and  the  other  four  fingers  are  performing 
extension/flexion  movements,  the  workspace  of  fingertips  is 
restricted  to  the  maximum  angles  of  joints.  Referring  to  inverse 


Fig.  6.  The  Output  Surface  of  Fuzzy  Inference  System  with  Two  Inputs  and 
49  Logic  Rules 
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Fig.  7.  Five-Fingered  Prosthetic  Hand  Grasping  a  Rectangular  Object 


side  length  and  length  of  the  target  rectangular  rod  are  0.010 
and  0.100  (m),  respectively.  The  relating  parameters  between 
the  global  coordinate  and  the  local  coordinates  are  defined  in 
Table  III.  Moreover,  in  this  work,  we  assumed  that  each  link 


TABLE  II 

Parameter  Selection  of  the  Smart  Hand 


Parameters 

Values 

Thumb 

Time  ( t0,tf  )* 

Desired  Initial  Position  (X^,Yq)** 

Desired  Final  Position  (Xj,Yp** 

Desired  Initial  Velocity  (Xq,Yq)* 

Desired  Final  Velocity  (Xj,Yf)* 

Length  (L^L^) 

Mass  (mj.m^) 

0,  20  (sec) 

0.035,  0.060  (m) 

0.0495,  0.060  (m) 

0,  0  (m/s) 

0,  0  (m/s) 

0.040,  0.040  (m) 

0.043,  0.031  (kg) 

Index  Finger 

Desired  Initial  Position  (X^Yj)** 

Desired  Final  Position  (X^,Yf)** 

Length  (L\,L\,L\) 

Mass  ( m\,m2,m |) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 
0.045,  0.025,  0.017  (kg) 

Middle  Finger 

Desired  Initial  Position  (X™,Y™)** 
Desired  Final  Position  (X™,Y™)** 

Length  (L™,L"\L™) 

Mass  (jn™  ,rn™  ,rn™) 

0.065,  0.080  (m) 

0.005,  0.060  (m) 

0.044,  0.044,  0.033  (m) 
0.050,  0.028,  0.017  (kg) 

Ring  Finger 

Desired  Initial  Position  (X^,Yq)** 

Desired  Final  Position  (Xf,Yf)** 

Length  (LJ,L£,L£) 

Mass  (m\,m2,m3) 

0.065,  0.080  (m) 

0.010,  0.060  (m) 

0.040,  0.040,  0.030  (m) 
0.041,  0.023,  0.014  (kg) 

Little  Finger 

Desired  Initial  Position  (X*,Yj)** 

Desired  Final  Position  ( Xlf,Y J)** 

Length  (L[,Ll2,Ll3) 

Mass  (m[,ml2,ml3) 

0.055,  0.080  (m) 

0.020,  0.060  (m) 

0.036,  0.036,  0.027  (m) 
0.041,  0.023,  0.014  (kg) 

*A11  fingers  use  same  parameters 
**A11  parameters  are  in  local  coordinates 

of  all  fingers  is  a  circular  cylinder  with  the  radius  ( R )  0.010 
(m),  so  the  inertia  IJzzk  of  each  link  k  of  all  fingers  j  (=  t,  i , 

TABLE  III 

Parameter  Selection  of  Conversion  between  Global  and  Local 
Coordinates 


Parameters 

Values 

Rotating  a 

90  (deg) 

Rotating  (5 

45  (deg) 

Translating  d1 

(0.035,  0,  0)  (m) 

Translating  dm 

(0.040,  0,  -0.020)  (m) 

Translating  dr 

(0.035,  0,  -0.040)  (m) 

Translating  d1 

(0.025,  0,  -0.060)  (m) 

m,  r  and  l)  can  be  calculated  as 

:Lk  =  \mkR2  +  \mkLC  ■  (22) 

All  initial  actual  angles  are  zero  and  the  diagonal  coefficients, 
Kp ,  Ki  and  Kd  ,  for  the  PD  or  PID  controller  alone  are  arbi¬ 
trarily  chosen  as  100.  From  the  derived  dynamic  and  control 
models,  after  the  parameters  (Kp  and  Kd)  are  selected,  the 
control  signal  u  and  torque  r  can  be  calculated. 

Figure  8  shows  tracking  errors  (e\  and  e\  on  the  left 


Time  (sec)  Time  (sec) 


Fig.  8.  Tracking  Errors  (Left)  and  Desired/Actual  Angles  (Right)  of  Joints 
1  (Upper)  and  2  (Lower)  for  Two-Link  Thumb  Using  PD  (Dash  Line),  PID 
(Dot  Line)  and  Fuzzy  Logic  Based  PD  (Solid  Line)  Controllers 

column)  and  desired/actual  angles  (q\  and  q\  on  the  right 
column)  of  joints  1  (the  top  row)  and  2  (the  bottom  row) 
for  two-link  thumb  using  PD  (dash  line),  PID  (dot  line)  and 
fuzzy  logic  based  PD  (solid  line)  controllers.  The  tracking 
errors  for  PD  control  are  convergent  within  5  seconds  without 
overshooting,  but  PID  control  acts  longer  (approximate  10  sec¬ 
onds)  with  overshooting  and  oscillation.  The  proposed  hybrid 
fuzzy  logic  PD  control  using  parameters  Kp  E  [50, 1000]  and 
Kd  G  [50,  500]  provides  faster  convergence  than  both  PD 
and  PID  controllers.  To  further  study  whether  the  parameter 
range  influences  tracking  errors,  we  found  that  the  larger 
parameter  range,  the  faster  convergent  speed  after  altering 
Kp  E  [50,  2000]  without  additional  computational  time.  We 
also  used  fuzzy  logic  controller  with  two  inputs  (error  and  error 
rate)  and  one  output  (control  signal),  but  the  control  system 
could  not  obtain  convergent  tracking  errors  (data  not  shown). 
These  data  suggest  that  fusion  of  soft  computing  (SC)  and 
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Fig.  9.  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1 
(Upper)  and  2  (Lower)  for  Two-Link  Thumb  Using  PD  (Dash  Line),  PID  (Dot 
Line)  and  Fuzzy  Logic  Based  PD  (Solid  Line)  Controllers  in  0.01  Second 


hard  control  (HC)  is  superior  to  either  HC  or  SC  methodology 
alone. 

The  time- variant  computed  control  signals  (u\  and  u\)  and 
torques  (r-f  and  r|)  for  two-link  thumb  within  the  first  0.01 
second  are  shown  in  Figure  9,  suggesting  that  the  presented 
hybrid  fuzzy  logic  PD  controller  requires  more  power  (torque) 
than  PD  and  PID  controllers  in  order  to  obtain  faster  conver¬ 
gent  tracking  errors. 

Similarly,  Figures  10,  12,  14  and  16  show  tracking  errors 
and  desired/actual  angles  for  all  remaining  three-link  fingers. 
Figures  11,  13,  15  and  17  show  control  signals  and  torques 
for  the  mapping  three-link  fingers. 


Fig.  1 1 .  Control  Signals  (Left)  and  Actuated  Torques  (Right)  of  Joints  1  (Top), 
2  (Middle)  and  3  (Bottom)  for  Three-Link  Index  Finger  Using  PD  (Dash  Line), 
PID  (Dot  Line)  and  Fuzzy  Logic  Based  PD  (Solid  Line)  Controllers  in  0.01 
Second 
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Fig.  12.  Tracking  Errors  (Left)  and  Desired/Actual  Angles  (Right)  of  Joints 
1  (Top),  2  (Middle)  and  3  (Bottom)  for  Three-Link  Middle  Finger  Using 
PD  (Dash  Line),  PID  (Dot  Line)  and  Fuzzy  Logic  Based  PD  (Solid  Line) 
Controllers 
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Fig.  10.  Tracking  Errors  (Left)  and  Desired/Actual  Angles  (Right)  of  Joints  1 
(Top),  2  (Middle)  and  3  (Bottom)  for  Three-Link  Index  Finger  Using  PD  (Dash 
Line),  PID  (Dot  Line)  and  Fuzzy  Logic  Based  PD  (Solid  Line)  Controllers 


V.  Conclusions  and  Future  Work 

For  a  five-fingered  smart  prosthetic  hand,  a  hybrid  control 
technique  involving  soft  control  with  adaptive  neuro-fuzzy 
inference  system  (ANFIS)  and  fuzzy  logic  (FL)  and  hard 
control  with  proportional-derivative  (PD)  was  presented.  The 
ANFIS  is  used  for  inverse  kinematics  and  FL  is  used  for 
tuning  the  PD  parameters  with  two  input  layers  (error  and 


error  change)  using  7  triangular  membership  functions  and 
49  fuzzy  logic  rules.  Simulation  results  with  FL-tuned  PD 
(hybrid)  controller  showed  superior  performance  compared  to 
the  PD  and  PID  control  alone.  Work  is  underway  to  extend  this 
methodology  to  a  real-time  implementation  on  the  prototype 
of  a  prosthetic  hand. 
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Abstract —  A  control  strategy  for  achieving  object  precision 
grasping  by  a  prosthetic  hand  is  proposed.  The  control 
strategy  is  based  on  defining  virtual  spring-damper  between 
two  finger  tips  and  damping  force  at  each  finger  joint.  It  is 
shown  that  the  proposed  control  strategy  provides  a 
satisfactory  performance  in  precision  grasping  of  a  prosthetic 
hand,  without  the  need  for  additional  complexity  regarding 
equations  for  inverse  kinematics,  or  inverse  dynamics,  and  the 
information  on  tactile  or  force  sensing  or  even  object  shape. 

I.  Introduction 

uman  hand  is  one  of  the  most  important  and  complex 
parts  of  the  body,  which  has  the  ability  to  handle 
different  tasks.  Loss  of  hand  can  highly  affect  the  quality  of 
life;  hence  there  is  a  high  demand  among  amputees  for 
prosthetic  hands.  The  goal  is  to  design  an  anthropomorphic 
prosthetic  hand  which  is  capable  to  be  controlled  through 
mind  and  has  functionality  close  to  normal  human  hand. 

The  muscle  movements  in  humans  are  controlled  by 
electromyographic  (EMG)  signal  which  comes  from  brain  to 
the  body.  In  case  of  a  lower  hand  amputee  the  remaining 
part  of  muscles  are  capable  to  measure  EMG  and  through  it 
find  the  subject’s  intention. 

Grasping  can  be  categorized  into  two  main  groups: 
precision  and  power  grasping.  In  precision  grasping  the 
object  is  held  by  tips  of  the  fingers,  while  in  power  grasping, 
the  whole  finger  is  active  and  in  contact  with  the  object  [1]. 
Our  previous  work  [2]  addressed  the  problem  of  power 
grasping  and  here  we  study  the  problem  of  precision 
grasping  of  a  prosthetic  hand. 

Many  control  methods  require  the  knowledge  of  the  shape 
of  the  object.  For  humans  this  information  is  available  by 
visual  feedback  from  eyes,  while  in  case  of  a  prosthetic  hand 
this  visual  information  is  not  directly  available  for  hand 
controller,  and  the  only  available  information  is 
electromyographic  (EMG)  signal  related  to  patient’s  arm 
muscle  activities.  However,  normally  the  EMG  signal  is  not 
available  for  all  individual  joints  and  besides,  due  to 
measurement  noise,  accessing  high  quality  EMG  signal  is 
hard  [3].  Moreover,  using  EMG  signal  to  control  all  the 
movements  requires  lot  of  attention  during  grasping  and 
leads  to  fatigue  for  the  amputees  [4].  Hence  it  is  required  for 
prosthetic  hand  to  be  semi-autonomous  which  means  a  part 
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of  command  information  will  be  provided  by  the  EMG 
signal  and  the  rest  of  the  required  command  should  be 
provided  automatically  by  hand  controller. 

Defining  finger  trajectory  without  the  knowledge  of  shape 
of  object  to  be  grasped  is  a  challenging  task  for  many  path 
planning  techniques.  For  multi  DOFs  robots  there  are  two 
common  methods  for  trajectory  planning  which  are  “inverse 
kinematics”  and  “inverse  dynamics”  [5] -[7].  Both  these 
methods  require  object  shape  and  are  based  on  solving 
optimization  problem  which  requires  high  computation, 
hence  they  are  hard  to  implement  for  real-time  applications. 

To  avoid  solving  the  path  planning  problem  for  prosthetic 
hands,  many  researchers  advocated  under- actuated 
mechanisms,  which  are  capable  of  adapting  to  object  shape 
mechanically  and  without  additional  computation  [8]- [10]. 
In  these  mechanisms,  the  number  of  actuators  is  less  than  the 
DOFs,  and  because  of  less  number  of  actuators  they  have 
less  weight.  However  fewer  actuators  result  in  less 
functionality,  because  fingers  joints  can’t  move 
independently. 

Arimoto  et  al.  [ip  used  “virtual  spring-damper 
hypothesis”  for  control  of  robotic  arm-hand  systems.  A 
similar  method  called  “virtual  model  control”  is  also 
suggested  by  J.Pratt  et  al.  [12]  used  for  walking  robots,  and 
it  is  based  on  defining  virtual  forces  between  two  points. 
Both  methods  are  based  on  the  use  of  Jacobian  matrix  to 
relate  task  space  movement  to  joint  space.  In  [12]  it  is 
shown  that  any  kind  of  force  can  be  defined  between  two 
points  and  the  other  study  [11]  shows  that  use  of  spring- 
damper  forces  will  result  in  human  like  movement.  From 
physiological  point  of  view,  human  skilled  multi-joint 
reaching  movement  has  these  characteristics:  1)  endpoint 
trajectory  become  a  quasi-straight  line  and  less  variable, 
2)velocity  profiles  of  the  endpoint  has  a  bell-shape,  and  3) 
joint  trajectories  are  rather  variable  from  trial  to  trial  [11]. 

In  this  paper,  a  new  control  scheme  is  proposed  that  can 
efficiently  address  the  problem  of  precision  grasping 
without  complete  knowledge  of  the  shape  of  the  object 
which  may  be  called  “blind  precision  grasping”  for 
prosthetic  hand.  The  proposed  method  is  based  on  the  works 
by  Arimoto  et  al.  [11]  using  virtual  spring-damper  (VSD) 
hypothesis  for  control  of  robotic  arm-hand  systems.  In  our 
paper,  we  use  the  above  mentioned  hypothesis,  in  particular 
for  the  precision  grasping  of  a  prosthetic  hand.  In  this 
method,  we  define  a  virtual  spring-damper  between  finger 
tip  and  desired  point  for  control  of  movement  of  fingers. 
Further,  in  this  method  there  is  no  need  to  introduce  any 
performance  indices  to  solve  inverse  kinematics  uniquely 
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and  Jacobian  pseudo-inverse  or  inverse  dynamics  which  are 
common  methods  to  define  trajectories  of  redundant  DOFs 
robots.  Besides,  in  the  present  method,  there  is  no  need  for 
any  information  on  tactile  or  force  sensing. 

The  paper  is  organized  as  follows.  In  Section  2  modeling 
of  prosthetic  hand  is  discussed,  Section  3  covers  virtual 
spring  damper  method.  Section  4  describes  control  strategy. 
Section  5  analyzes  the  efficiency  of  the  proposed  control 
scheme  using  numerical  simulation.  Finally,  conclusion  and 
discussion  are  presented  in  section  6. 

II.  Modeling  of  Prosthetic  Hand 

In  this  control  method,  controller  is  not  designed  based  on 
dynamic  model  of  the  system.  Instead,  kinematics  equation 
and  Jacobian  matrix  are  used  for  controller  design. 

A  model  of  a  robotic  hand  system  is  shown  in  Fig.l.  The 
model  consists  of  a  finger  with  3  DOF  which  represents  three 
joints  of  index  finger,  palm  and  a  finger  with  2DOF  which 
represents  thumb. 


In  this  paper  we  assume  the  following: 

1)  Movement  of  both  finger  and  object  are  confined  to  a  2 
dimensional  horizontal  plane,  and  therefore  there  is  no 
gravity  effect. 

2)  The  object  is  assumed  to  be  initially  stable  in  its 
position. 

3)  The  initial  movement  toward  object  is  handled  by 
amputee,  so  the  hand  is  close  enough  to  the  object  before 
grasping. 

The  position  of  the  tip  of  index  fingers  is  evaluated  as 
(see  Fig.  1): 

Xi  =  k i  cosqn  +  lt 2  cos^qa  +  qi2)  +  li3  cos (qn  + 
qi2 +qi3),  (1) 

Vi  =  hi  sinq n  +  hi  sin (qn  +  qi2)  +  li3  sin{qtl  + 
qi2+qi3),  (2) 


Based  on  above  equation  the  Jacobian  matrix  for  index 
finger  is  as: 


(5) 


(6) 


III.  Virtual  Spring  Damper  Method 

Virtual  model  control  was  first  proposed  by  J.  Pratt  et  al 
[12]  for  biped  walking  robot.  This  method  is  a  motion 
control  scheme  that  uses  simulations  of  virtual  components 
to  generate  desired  joint  torques  [12].  These  joints  produce 
the  same  effect  that  the  virtual  elements  placed  on  robot 
would  have  created;  hence  they  create  the  illusion  that  these 
virtual  elements  are  connected  to  the  real  robot.  Virtual 
elements  can  be  any  kind  of  real  physical  elements  such  as 
springs,  dampers,  gravity  fields,  nonlinear  fields  or  any 
other  components.  In  a  study  by  Arimoto  [11]  on  robotic 
hand  arm  system,  it  is  shown  that  using  a  virtual  spring 
damper  between  robot  end  effecter  and  desired  point,  and 
virtual  dampers  at  each  joint,  human  like  movement  can  be 
achieved. 

For  precision  grasping  by  a  prosthetic  hand,  one  of  the 
best  options  is  the  use  of  Virtual  Spring-Damper  (VSD) 
hypothesis.  Some  benefits  of  VSD  control  scheme  are  that  it 
has  a  simple  structure  and  requires  relatively  less 
computation.  Besides,  it  doesn’t  need  inverse  dynamics  to 
precisely  define  the  robot  movement.  Thus,  we  use  spring 
set  points  instead  of  commanded  movement  and  robot 
automatically  adapts  its  shape.  Since  finger  joints  at 
prosthetic  hand  work  as  virtual  dampers,  which  is  sensitive 
to  velocity  and  not  to  position,  they  don’t  have  a  forced 
shape,  instead  just  finger  tip  follow  a  defined  path  as  will  be 
discussed  more  in  control  strategy  section. 

The  joint  torques  to  virtual  forces  is  given  by: 

t  =JTF,  (7) 

where  x  is  the  torque,  and  F  is  the  force  due  to  virtual  spring 
damper  given  as 

F  =  -(f\ fkx  +  kAx),  (8) 


where,  lih  li2,  and  li3  are  lengths  of  index  finger  and  qilf  qi2, 
and  qi3  are  angles  of  each  corresponding  joint.  Similarly  the 
position  of  thumb  finger  is  evaluated  as: 
xt  =  ltl  cos  qtl  +  lt2  cos(qtl  +  qt2),  (3) 

Vt  =  hi  sin  qtl  +  lt2  sin(qtl  +  qt2),  (4) 

where,  ltl  and  lt2  are  lengths  of  thumb  finger  and  qti  and  qt2, 
are  angles  of  corresponding  joints. 


and 

T spring -damper  ~  ~J  T  kAx ),  (9) 

where  k  represents  the  stiffness  of  the  virtual  spring,  Ax  is 
distance  between  finger  tip  and  desired  point,  and  £  is  the 
damping  ratio.  The  damping  force  is  defined  at  each  joint  as 
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T joints  damping  ~Cq,  (10) 

where,  C  denotes  a  diagonal  positive  definite  matrix  as 
follows: 

C  =  fo  diagic^  ...,cn).  (11) 

Hence  control  signal  would  be  sum  of  these  two  terms: 
u  =  -Cq  -  JT(q)^Vkx  +  kAx).  (12) 

IV.  Control  Strategy 

Virtual  spring-damper  hypothesis  is  suitable  for  point  to 
point  control.  In  precision  grasping  two  approaches  can  be 
considered.  1)  Defining  a  virtual  spring  damper  between 
fingers  tip  and  geometrical  center  of  the  object,  which 
requires  information  about  the  object  position  and  shape, 
and  this  information  is  not  available  in  case  of  a  prosthetic 
hand  for  the  controller  which  is  used  in  [13]  2)  Defining  a 
virtual  spring  damper  between  tips  of  two  fingers,  then 
fingers  attract  together  and  grasp  the  object  in  between, 
without  exact  knowledge  of  object  position  and  shape.  In 
this  case  the  amputee  should  place  the  hand  close  to  the 
object  and  in  appropriate  position.  Besides,  a  virtual  damper 
force  is  considered  at  each  finger  joint.  The  latter  method  is 
used  and  physical  counterpart  of  virtual  forces  are  depicted 
at  Fig.  2. 


Virtual  Spring  Damper 


Fig.  2.  Physical  counterparts  of  the  virtual  forces 


Higher  values  of  k  (virtual  spring  stiffness)  results  in 
faster  movement  of  fingers  as  well  as,  higher  grasping  force. 
Thus  by  defining  k  proportional  to  EMG  signal,  amputee 
have  control  over  speed  of  movement  and  grasping  force. 
The  damping  coefficient  of  finger  joints,  can  change  the 
final  shape  of  fingers.  The  joints  with  lower  damping  tends 
to  move  more,  while  higher  damped  joints  move  more.  The 
appropriate  values  of  damping  are  evaluated  based  on  trial 
and  error  to  reach  positions  close  to  normal  hand  and  they 
are  held  constant  for  further  simulations. 

V.  Numerical  Simulation 

In  order  to  show  the  effectiveness  of  the  proposed  control 
strategy,  numerical  simulations  were  conducted  to  grasp  two 


different  objects,  based  on  the  physical  parameters  of  a  hand 
system  and  objects  summarized  in  Table  I. 

The  Adams  software  which  is  multi-body  dynamic 
simulation  software  is  used  for  numerical  analysis.  The 
software  is  capable  to  conduct  information  between  Matlab/ 
Simulink  software  environment,  hence  the  plant  is  modeled 
by  Adams  and  controller  is  implemented  in  Matlab/Simulnk. 


TABLE  I 

Parameters  Used  for  Simulation 


Parameter 

Value 

index  finger  link  1  length 

5  cm 

index  finger  link  2  length 

2.5  cm 

index  finger  link  3  length 

2.5  cm 

thumb  finger  link  1  length 

4  cm 

thumb  finger  link  2  length 

3  cm 

distance  between  thumb  and  index 

6  cm 

damping  at  joints 

0.01  kg/s 

virtual  damping  ratio 

1 

virtual  spring  stiffness 

50  N/m 

rectangular  object  width 

2  cm 

circular  object  radius 

2  cm 

Fig.  3.  Finger  movements  in  4  different  position  (0.25  sec.  intervals) 


For  first  simulation,  two  fingers  are  modeled  without  any 
object  in  between.  As  shown  in  Fig.  3,  two  fingers  come 
together,  and  final  position  is  close  to  normal  hand 
coordination. 


Fig.  4.  Rectangular  object  grasping  (0.25  sec  intervals) 


For  the  second  simulation,  a  rectangular  object  is  chosen 
to  be  grasped.  The  object  is  free  to  move  in  2  dimensional 
plane,  and  contact  and  friction  force  are  simulated  between 
finger  tip  and  the  object.  The  object  is  placed  at  arbitrary 
final  position  of  previous  experiment.  The  finger  movement 
at  0.25  sec  time  intervals  and  finger  tip  angles  relative  to 
palm  are  shown  respectively  in  Figs.  4  and  5. 
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Fig.  5.  Finger  tips  angle  at  rectangular  object  grasping 
(index  finger  solid  line  and  thumb  finer  dashed  line) 

As  it  is  shown  after  contact  with  object  at  approximately 
1.5  second  the  angles  are  not  changing  much.  The  small 
changes  are  due  to  object  movements  toward  left. 


thumb 


index  finger 

Fig.  6.  Circular  object  grasping  (0.25  sec  intervals) 


For  third  simulation  a  round  object  is  selected.  The  object 
is  not  subjected  to  any  constrain  in  2  dimensional  plane.  The 
contact  and  friction  force  are  defined  between  object  and 
finger  tip.  The  virtual  spring  coefficient  which  is 
proportional  to  EMG  signal  is  assumed  to  be  constant. 
Almost  after  1  sec,  the  fingers  contacted  with  the  object.  The 
finger  movement  at  0.25  sec  time  intervals  and  finger  tip 
angles  relative  to  palm  are  shown  respectively  in  Figs.  6  and 
7. 


Fig.  7.  Finger  tips  angle  at  circular  object  grasping 
(index  finger  solid  line  and  thumb  finer  dashed  line) 


Appropriate  object  position  and  friction  force  between 
fingers  and  object  are  important  parameters  that  help 
successful  grasping.  For  the  case  of  a  round  object  if  the 
object  has  inappropriate  position  or  friction  is  not  enough, 
the  grasping  might  be  unstable,  but  guarantied  successful 
grasping  under  all  conditions  require  information  about 
object  shape  and  position  which  are  not  available  for  case  of 
a  prosthetic  hand  hence  this  control  strategy  is  appropriate 
for  most  of  the  daily  activities. 


VI.  Conclusion 

Based  on  virtual  spring  damper  hypothesis,  a  control 
strategy  for  precision  grasping  of  a  prosthetic  hand  was 
proposed.  The  controller  doesn’t  require  extensive 
computation  of  inverse  kinematic  or  inverse  dynamics  to 
describe  finger  movement  and  hence  the  method  is 
appropriate  for  real-time  applications.  The  control  strategy  is 
independent  of  object  shape  or  force  and  contact  sensor.  By 
numerical  simulation,  the  capability  and  effectiveness  of 
control  strategy  are  shown.  The  same  strategy  was  used  for 
two  different  objects  and  grasping  was  successful. 

The  future  work  will  focus  on  precision  grasping  by 
gravity  cancelation  and  analysis  in  three-dimensional  space 
and  experimental  implementation  with  EMG  signals. 
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Fusion  of  Fuzzy  Logic  and  Proportional-Derivative  Control 
for  a  Five-Fingered  Smart  Prosthetic  Hand 


Abstract 

A  hybrid  of  soft  control  technique  of  adaptive  neuro-fuzzy 
inference  system  (ANFIS)  and  fuzzy  logic  (FL)  and  hard  control 
technique  of  proportional-derivative  (PD)  for  a  five-fingered, 
smart  prosthetic  hand  is  presented.  The  ANFIS  is  used  for 
inverse  kinematics  and  FL  is  used  for  tuning  the  PD  parameters 
with  two  input  layers  (error  and  error  change)  using  7  triangular 
membership  functions  and  49  fuzzy  logic  rules.  Simulation 
results  with  FL-tuned  PD  controller  exhibit  superior  performance 
compared  to  the  PD  and  PID  control  alone. 


Methods 


Figure  1.  Schematic  Diagram  of  Prosthetic  Hand  Technology:  The  overall  system 
consists  of  electromyographic  (EMG)  signal  acquisition  from  the  user  arm  for  surface  or 
implanted  electrodes  (in  the  implanted  case  we  focus  on  biocompatibility  based  on  nano¬ 
materials  research).  The  EMG  signal  is  then  processed  for  feature  extraction  and 
classification  or  identification  of  EMG  signal  to  correspond  to  different  motions  of  the 
prosthetic  hand.  The  classified  signal  is  then  used  to  control  the  prosthetic  hand  using 
actuators  and  driving  mechanisms. 

Figure  2.  Principle  behind 
Fusion  of  Soft  and  Hard 
Computing:  The  present 
research  focuses  on 
developing  intelligent 
autonomous  strategies  for 
EMG  signal,  extraction, 
analysis  and  control 
prosthetics  by  fusion  of  soft 
computing  (SC)  techniques 
comprising  neural  network 
(NN),  fuzzy  logic  (FL),  and 
genetic  algorithm  (GA)  and 
hard  control  (HC)  techniques, 
such  as  proportional-derivative 
(PD),  optimal  control,  adaptive 
control  and  so  on.  We  present 
the  integration  of  SC  and  HC 
to  solve  problems  that  cannot 
be  solved  satisfactorily  by 
using  either  HC  or  SC 
methodology  alone  with 
specific  applications  to 
prosthetics. 
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Figure  3.  Block  Diagram  of  the  Presented  Hybrid  Fuzzy  Logic- 
Based  Proportional-Derivative  (PD)  Controller  for  a  Five-Fingered 
Prosthetic  Hand:  Errors  and  error  changes  are  calculated  by  actual 
and  desired  angles,  which  are  based  on  adaptive  neuro-fuzzy  inference 
system  (ANFIS)  trajectory  planner.  Then  fuzzy  logic  controller  tunes  all 
parameters  of  closed-loop  PD  control  so  that  the  required  torque  of  the 
prosthetic  hand  nonlinear  system  is  computed  by  control  input. 
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Figure  4.  ANFIS  Architecture:  (a)  A  Two  Input  First-Order  Sugeno 
Fuzzy  Model  with  Two  Rules  (b)  Equivalent  ANFIS  Structure.  The 

inverse  kinematics  problems  are  solved  by  using  adaptive  neuro-fuzzy 
inference  system  (ANFIS)  method  where  the  input  of  fuzzy-neuro 
system  is  the  Cartesian  space  and  the  output  is  the  joint  space.  ANFIS 
is  a  fuzzy  inference  system  implemented  in  the  framework  of  adaptive 
networks  which  provides  the  best  optimization  algorithm  to  find 
parameters  in  order  to  fit  the  data. 


Results 


Table  1.  Parameter  Selection  of  Conversion 
between  Global  and  Local  Coordinates 
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Figure  5.  Relationship  between  Global  and  Local 
Coordinates:  Local  coordinate  xf-/-z!  of  thumb  can  be  reached 
by  rotating  through  angles  a  and  (3  to  XG  and  Y°  of  global 
coordinate,  subsequently.  Local  coordinate  x'-y'-z'  of  index  finger 
can  be  obtained  by  rotating  through  angle  a  to  Xs  and  then 
translating  a  vector  d'  of  the  global  coordinate. 


Table  2.  A  Complete  Fuzzy  Logic  Rule  Base 
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Table  3.  Parameter  Selection  of  the  Smart 
Hand 


Figure  6.  The  Structural  Characteristics  of  Proposed  Fuzzy 
Inference  System:  Two  inputs  (error  and  error  change)  on  the 
left  and  one  output  (tuned  KP)  on  the  right  with  49  logic  rules 
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Figure  7.  Each  of  Two  Inputs  (Upper  Panel)  and  One  Output 
(Lower  Panel)  Uses  7  Triangular  Membership  Functions 
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Figure  8.  The  Output  Surface  of  Fuzzy  Inference  System 
with  Two  Inputs  and  49  Logic  Rules 


Figure  9.  Five-Fingered 
Prosthetic  Hand 
Grasping  a  Rectangular 
Object 
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Figure  10.  Tracking 
Errors  (Left)  and  Desired 
and  Actual  Angles  (Right) 
of  Joints  1  (Upper)  and  2 
(Lower)  for  Two-Link 
Thumb  Using  PD  (Dash 
Line),  PID  (Dot  Line)  and 
Fuzzy  Logic  Based  PD 
(Solid  Line)  Controllers 


Figure  11.  Tracking  Errors 
(Left)  and  Desired  and 
Actual  Angles  (Right)  of 
Joints  1  (Top),  2  (Middle) 
and  3  (Bottom)  for  Three- 
Link  Index  Finger  Using 
PD  (Dash  Line),  PID  (Dot 
Line)  and  Fuzzy  Logic 
Based  PD  (Solid  Line) 
Controllers 


Conclusions  and  Future  Work 

1.  For  a  five-fingered,  smart  prosthetic  hand,  a  hybrid  control 
technique  involving  soft  control  with  adaptive  neuro-fuzzy 
inference  system  (ANFIS)  and  fuzzy  logic  (FL)  and  hard  control 
with  proportional-derivative  (PD)  was  presented. 

2.  The  ANFIS  is  used  for  inverse  kinematics  and  FL  is  used  for 
tuning  the  PD  parameters  with  two  input  layers  (error  and  error 
change)  using  7  triangular  membership  functions  and  49  fuzzy 
logic  rules. 

3.  Simulation  results  with  FL-tuned  PD  controller  showed  superior 
performance  compared  to  the  PD  and  PID  control  alone. 

4.  Work  is  underway  to  extend  this  methodology  to  a  real-time 
prosthetic  hand. 
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ABSTRACT 

A  control  scheme  for  achieving  object  power  grasping  by 
a  prosthetic  hand  is  proposed.  The  control  scheme  is 
based  on  defining  virtual  spring-damper  between  finger 
tip  and  desired  point,  and  a  semi  circular  path  for  finger 
tip.  It  is  shown  that  the  suggested  control  scheme  provides 
satisfactory  performance  in  power  grasping  of  prosthetic 
hand,  without  the  need  for  additional  complexity 
regarding  equations  for  inverse  kinematics,  or  inverse 
dynamics,  and  the  information  on  tactile  or  force  sensing 
or  even  object  shape. 

KEY  WORDS 

Medical  Robotics,  Control,  Power  Grasping,  Prosthetic 
Hand  Control. 

1.  Introduction 

Human  hand  is  one  of  the  most  important  and  complex 
parts  of  the  body,  which  has  the  ability  to  handle  different 
tasks.  The  ultimate  goal  of  robotic  hand  is  to  achieve  the 
functionality  of  a  human  hand.  In  the  past  three  decades, 
there  have  been  numerous  investigations  to  achieve 
dexterity  and  ability  of  human  hand,  especially  in  the 
fields  of  humanoid  robotics  and  prosthetic  hand  [1-6].  In 
spite  of  all  these  advances  in  this  field,  the  current  state  of 
research  on  prosthetic  hands  is  far  from  that  objective  of 
achieving  the  functionality  of  human  hand. 
Commercially  available  prosthetic  hands  have  very 
limited  functionality  and  they  are  just  simple  grippers. 
The  present  research  on  prosthetic  hands  involves 
complex  control  schemes  to  achieve  the  most  important 
functions  of  the  hand  [7-8]. 

Grasping  can  be  categorized  into  two  main  groups: 
precision  and  power  grasping.  In  precision  grasping  the 
object  is  held  by  tips  of  the  fingers,  while  in  power 
grasping,  the  whole  the  finger  is  active  and  in  contact 
with  the  object  [9].  Many  research  works  addressed  the 
precision  grasping  problem  [12-15]  mainly  because  of  the 
well-established  techniques  for  control  of  end  effector; 
however  the  problem  of  power  grasping  is  not  studied  in 
depth. 

Many  control  methods  require  the  knowledge  of  the  shape 
of  the  object.  For  humans  this  information  is  available  by 


visual  feedback  from  eyes,  while  in  case  of  a  prosthetic 
hand  this  visual  information  is  not  directly  available  for 
hand  controller,  and  the  only  available  information  is 
electromyographic  (EMG)  signal  related  to  patient’s  arm 
muscle  activities.  However,  normally  the  EMG  signal  is 
not  available  for  all  individual  joints  and  besides,  due  to 
measurement  noise,  accessing  high  quality  EMG  signal  is 
hard  [10].  Moreover,  using  EMG  signal  to  control  all  the 
movements  requires  lot  of  attention  during  grasping  and 
leads  to  fatigue  for  the  amputees  [11].  Hence  it  is  required 
for  prosthetic  hand  to  be  semi-autonomous  which  means  a 
part  of  command  information  will  be  provided  by  the 
EMG  signal  and  the  rest  of  the  required  command  should 
be  provided  automatically  by  hand  controller. 

Defining  finger  trajectory  without  the  knowledge  of  shape 
of  object  to  be  grasped  is  a  challenging  task  for  many  path 
planning  techniques.  For  multi  DOFs  robots  there  are  two 
common  methods  for  trajectory  planning  which  are 
“inverse  kinematic”  and  “inverse  dynamic”  [12-15].  Both 
these  methods  require  object  shape  and  are  based  on 
solving  optimization  problem  which  requires  high 
computation,  hence  they  are  hard  to  implement  for  real¬ 
time  applications. 

To  avoid  solving  the  path  planning  problem  for  prosthetic 
hands,  many  researchers  advocated  under-actuated 
mechanisms,  which  are  capable  of  adapting  to  object 
shape  mechanically  and  without  additional  computation 
[16-18].  In  these  mechanisms,  the  number  of  actuators  are 
less  than  the  DOFs,  and  because  of  less  actuators  they 
have  less  weight.  However  fewer  actuators  result  in  less 
functionality,  because  fingers  joints  can’t  move 
independently. 

Arimoto  et  al.  [19]  used  “virtual  spring-damper 
hypothesis”  for  control  of  robotic  arm-hand  systems.  A 
similar  method  called  “virtual  model  control”  is  also 
suggested  by  J.Pratt  et  al.  [20]  used  for  walking  robots, 
and  it  is  based  on  defining  virtual  forces  between  two 
points.  Both  methods  are  based  on  the  use  of  Jacobian 
matrix  to  relate  task  space  movement  to  joint  space.  In 
[20]  it  is  shown  that  any  kind  of  force  can  be  defined 
between  two  points  and  the  other  study  [19]  shows  that 
use  of  spring-damper  forces  will  result  in  human  like 
movement.  From  physiological  point  of  view,  human 
skilled  multi -joint  reaching  movement  has  these 
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characteristics  that  1)  endpoint  trajectory  become  a  quasi¬ 
straight  line  and  less  variable,  2)velocity  profiles  of  the 
endpoint  has  a  bell-shape,  and  3)  joint  trajectories  are 
rather  variable  from  trial  to  trial  [19]. 

In  this  paper,  a  new  control  scheme  is  proposed  that  can 
efficiently  address  the  problem  of  power  grasping  without 
complete  knowledge  of  the  shape  of  the  object  which  may 
be  called  “blind  power  grasping”  for  prosthetic  hand.  The 
proposed  method  is  based  on  the  works  by  Arimoto  et  al. 
[19]  using  virtual  spring-damper  (VSD)  hypothesis  used 
for  control  of  robotic  arm-hand  systems.  In  our  paper,  we 
use  the  above  mentioned  hypothesis,  in  particular  for  the 
power  grasping  of  a  prosthetic  hand.  In  this  method,  we 
define  a  virtual  spring-damper  between  finger  tip  and 
desired  point  for  control  of  movement  of  fingers.  Further, 
in  this  method  there  is  no  need  to  introduce  any 
performance  indices  to  solve  inverse  kinematics  uniquely 
and  Jacobian  pseudo-inverse  or  inverse  dynamics  which 
are  common  methods  to  define  trajectories  of  redundant 
DOFs  robots.  Besides,  in  the  present  method,  there  is  no 
need  for  any  information  on  tactile  or  force  sensing. 

The  paper  is  organized  as  follows.  In  Section  2  modeling 
of  prosthetic  hand  is  discussed,  Section  3  covers  virtual 
spring  damper  method.  Section  4  describes  control 
strategy.  Section  5  analyzes  the  efficiency  of  the  proposed 
control  scheme  using  numerical  simulation.  Finally, 
conclusion  and  discussion  are  presented  in  section  6. 


2.  Modeling  of  Prosthetic  Hand 

In  this  control  method,  controller  is  not  derived 
directly  from  dynamic  model  of  the  system.  Kinematics 
equation  and  Jacobian  matrix  are  the  required  for 
controller  design. 

A  model  of  a  robotic  hand  system  is  shown  in  Fig.l. 
The  model  consists  of  a  finger  with  3DOF  which 
represents  three  joints  of  a  finger  and  palm. 


Fig.  1:  Model  of  a  Robotic  Hand  System. 

In  this  paper  we  assume  the  following: 

1-  Movement  of  both  finger  and  object  are  confined  to  a 
two  dimensional  horizontal  plane,  and  therefore  there  is 
no  gravity  effect. 

2-  The  object  is  assumed  to  be  initially  stable  in  its 
position. 


3-  The  initial  movement  toward  object  is  handled  by 
amputee,  so  the  hand  is  close  enough  to  the  object  before 
grasping. 

The  position  of  the  tip  of  the  finger  is  evaluated  as  (see 
Figure  1): 

x  =  l±  cos  qx  +  l2  cos(q1  +  q2 ) 

+  l3  cos(q1  +q2+  q3)  (!) 

y  =  k  sin  ^  +  l2  sin(q1  +  q2) 

+  l3  sin(qt  +  q2  +  q3)  (2) 

where,  lh  /2,  and  l3  are  lengths  of  each  finger  and  qh  q2, 
and  q3  are  angles  of  each  corresponding  joint. 

Based  on  above  equation  the  Jacobian  matrix  is  as: 


j  _  pn  j  12  J 13  (3) 

U21  J22  J23] 

ill  =  -li  Sin  q1  -  l2  sin (qx  +  q2)  - 
l3  sin(q1  +q2  +  q3 ),  (4) 

A2  =  ~h  sin (<h  +  q2)  -  l3  sin(q1  +  q2  +  q3),  (5) 

A2  =  ~h  sin(q1  +  q2)  -  l3  sin(q1  +  q2  +  q3),  (6) 

7 13  =  ~h  sin(<h  +q2+  <7s)  -  (7) 

721  =  h  cos  qt  +  l2  cos (qx  +  q2)  + 
h  cos{q1  +  q2+  q3)  ,  (8) 

7'22  =  h  cos(qt  +  q2)  +  l3  cos (tfr  +  q2  +  q3 ),  (9) 

723  =  *3  cos (<7l  +  <?2  +  93)-  (10) 


3.  Virtual  Spring-Damper  Method 

“Virtual  model  control”  is  a  motion  control  scheme  that 
uses  simulations  of  virtual  components  to  generate 
desired  joint  torques  [20].  These  joints  produce  the  same 
effect  that  the  virtual  elements  placed  on  robot  would 
have  created;  hence  they  create  the  illusion  that  these 
virtual  elements  are  connected  to  the  real  robot.  Virtual 
elements  can  be  any  kind  of  real  physical  elements  such 
as  springs,  dampers,  gravity  fields,  nonlinear  fields  or  any 
other  components. 

Virtual  model  control  was  proposed  by  J.  Pratt  et  al  [20] 
for  biped  walking  robot.  In  a  study  by  Arimoto  [19]  on 
robotic  hand  arm  system,  it  is  shown  that  using  a  virtual 
spring  damper  between  robot  end  effector  and  desired 
point,  and  virtual  dampers  at  each  joint,  human  like 
movement  can  be  achieved. 

For  power  grasping  by  a  prosthetic  hand,  one  of  the  best 
options  is  the  use  of  Virtual  Spring-Damper  (VSD) 
hypothesis.  Some  benefits  of  VSD  control  scheme  are  that 
it  has  a  simple  structure  and  requires  relatively  less 
computation.  Besides,  it  doesn’t  need  inverse  dynamics  to 
precisely  define  the  robot  movement.  Thus,  we  use  spring 
set  points  instead  of  commanded  movement  and  robot 
automatically  adapts  its  shape.  Since  finger  joints  at 
prosthetic  hand  work  as  virtual  dampers,  which  is 
sensitive  to  velocity  and  not  to  position,  they  don’t  have  a 
forced  shape,  instead  just  finger  tip  follow  a  defined  path 
as  will  be  discussed  more  in  control  strategy  section. 

The  joint  torques  to  virtual  forces  is  given  by: 
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T  =  ]TF, 


(11) 


where  x  is  the  torque,  and  F  is  the  force  due  to  virtual 
spring  damper  given  as 

F  =  —  (<fV/cx  +  kAx),  (12) 

and 

T spring -damper  ~  ~J  (<fV/cX  +  kAx),  (12) 

where  k  represents  the  stiffness  of  the  virtual  spring,  Ax  is 
distance  between  finger  tip  and  desired  point,  and  £  is  the 
damping  ratio.  The  damping  force  is  defined  at  each  joint 
as 


T joints  damping 


=  —Cq, 


(14) 


where,  C  denotes  a  diagonal  positive  definite  matrix  as 
follows: 


c  =  to  diagici,  ...,cn). 


(15) 


Hence  control  signal  would  be  sum  of  these  two  terms 

u  =  —Cq  -/r(q)(<fV/cx  +  kAx).  (16) 

Higher  values  of  k  result  in  more  accurate  and  faster 
response  to  the  desired  point  and  higher  C  provides  more 
stability.  Thus  k  and  C  are  chosen  as  design  variables. 


4.  Control  Strategy 

Virtual  spring-damper  hypothesis  is  suitable  for  point  to 
point  control.  Defining  the  desired  trajectory  as  a 
semicircle  (in  order  to  have  a  full  grasp  of  the  object) 
given  by, 

xd  =  Oi  +  h  +  h)  cos(t),  (17) 

Vd  =  Qi  +  h  +  h)  sin(  t),  (18) 

Where  t  is  proportional  to  EMG  signal  which  is  scaled  to 
change  between  0<t<7i. 

As  shown  in  Fig.  2,  after  passing  this  semi-circle,  finger 
tip  goes  toward  center  to  make  a  tighter  grasping.  This  is 
achieved  by  defining  a  desired  point  close  to  center. 

As  mentioned  earlier,  the  goal  is  not  exactly  following  the 
defined  path.  If  the  object  is  big,  due  to  contact  of  hand 
and  object,  it  would  be  impossible  to  follow  exact  path 
and  following  this  path  is  just  to  achieve  grasping. 

Fig.  3  illustrates  the  physical  counterparts  of  the  virtual 
forces  for  control  strategy  and  Fig.  4  shows  structure  of 
the  proposed  control  system.  As  shown,  the  command 
force  comes  from  EMG  signal,  and  controller  provides 
the  movement  for  hand  which  has  dynamic  interaction 
with  the  object. 


Fig.  2:  Semi-Circle  Path  of  Finger  Tip 


Desired  Point 


Fig  3:  Physical  Counterparts  of  the  Virtual  Forces 


Fig.  4:  Control  Diagram  of  Prosthetic  Hand  System 


5.  Numerical  Simulation 

In  order  to  show  the  effectiveness  of  the  proposed  control 
strategy,  numerical  simulations  were  conducted  to  grasp 
three  different  objects,  based  on  the  physical  parameters 
of  a  hand  system  and  objects  summarized  in  Table  1. 

In  order  to  simulate  dynamics  of  the  hand  and  interaction 
with  object,  Adams  software  is  used.  This  software  is  a 
multi-body  analysis  simulation  program  that  solves  the 
rigid  body  dynamic  equilibrium  equations  and  directly 
interfaces  with  Matlab/Simulink  software  in  order  to 
implement  controller.  The  contact  between  object  and 
hand  is  modeled  and  three  sets  of  simulations  with 
different  objects  are  performed.  In  all  three  simulations 


Page  351  of  531 


the  same  control  strategy  is  used  which  shows  controller 
can  handle  grasping  without  information  about  physical 
parameters  of  object.  Fingers  and  objects  are  assumed  to 
be  rigid.  In  these  simulations  the  EMG  signal  is  assumed 
to  increase  linearly  with  time. 

Table  1.  Parameters  used  for  simulation 


Index  finger  link  1_ 5  cm 


Index  finger  link  2 

2.5  cm 

Index  finger  link  3 

2.5  cm 

Damping  at  joints 

0.01  kg/s 

Virtual  damping  ratio  (c) 

1 

Virtual  spring  stiffness  (k) 

50  N/m 

Rectangular  object  width 

3  cm 

Circular  object  radius 

3  cm 

Star  shape  object  outer  radius 

2  cm 

5.1  Simulation  One 

For  first  simulation  a  rectangular  (cubic)  object  is  used, 
and  as  mentioned  earlier  the  movement  is  restricted  to  2D 
movement.  Object  is  not  moving  initially.  Hand  starts 
movement  from  open  finger  configuration.  The  hand 
positions  at  1  second  time  interval  are  shown.  Fig.  5 
shows  finger  tip  angle  with  respect  to  palm. 

As  it  is  shown  in  Fig.  6  the  grasping  is  accomplished 
successfully. 


Fig.  5:  Finger’s  Tip  Angle  at  Rectangular  Object 
Grasping 


5.2  Simulation  Two 

For  the  second  simulation  a  glass  (circular  object)  is  used. 
The  control  parameters  are  identical  to  previous 
simulation. 

As  shown  in  Fig.  7  the  grasping  is  done  successfully  and 
the  hand  positions  for  1  second  time  interval  are  depicted. 
Similarly  the  finger  tip  angle  respect  to  palm  is  shown  in 
Fig.  8. 

Regardless  of  object  shape,  by  use  of  proposed  control 
scheme  the  hand  can  successfully  grasp  objects. 


Fig.  6:  Rectangular  Object  Grasping  (1  Sec  Intervals) 
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Fig.  7  (a-h)  :  Grasping  of  a  Glass  (1  sec.  Intervals) 


Page  353  of  531 


Fig.  8:  Finger’s  Tip  Angle  at  Grasping  a  Glass 


5.3  Simulation  Three 

For  the  third  simulation  a  star  shape  object  is  used.  The 
control  parameters  are  identical  to  previous  simulation. 

As  shown  in  Fig.  9  the  grasping  is  done  successfully  and 
the  hand  positions  for  1  second  time  interval  are 
illustrated.  Final  position  of  grasping  in  3D  (isometric) 
view  is  depicted  in  Fig.  10.  Similarly  the  finger  tip  angle 
respect  to  palm  is  shown  in  Fig.  11. 

The  complicated  shape  of  object  shows  that  controller  is 
able  to  handle  grasp  for  wide  variety  of  objects,  without 
information  of  object  shape. 


(C) 


Fig.  9  (a-h)  :  Grasping  of  Star  Shape  Object  (1  sec. 
Intervals) 
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Fig.  10  :  3D  (Isometric)  View  of  Star  Shape  Object  After 
Grasping 


Fig.  11:  Finger’s  Tip  Angle  at  Star  Shape  Object  Grasping 


6.  Conclusion 

In  this  paper  a  control  scheme  for  achieving  object  power 
grasping  by  a  prosthetic  hand  was  proposed.  The  control 
scheme  was  based  on  defining  virtual  spring-damper 
between  finger  tip  and  desired  point,  as  well  as,  damping 
at  finger  joints,  and  a  semi  circular  path  for  finger  tip.  It 
was  shown  through  three  sets  of  numerical  simulations  for 
different  objects  that  the  proposed  control  scheme  can 
effectively  be  used  for  power  grasping  of  prosthetic  hand. 
It  is  shown  that  regardless  of  object  shape  proposed 
control  method  is  successful  for  wide  variety  of 
situations.  The  proposed  method  did  not  rely  on  complex 
equations  of  inverse  kinematics,  or  inverse  dynamic  and 
hence  this  method  is  better  suitable  for  real-time 
applications.  The  immediate  future  investigation  is  the 
application  of  the  proposed  method  to  precision  grasping 
of  prosthetic  hand. 
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Abstract  -  This  paper  presents  a  novel  approach  for 
hierarchical  multi-level  motor  control  for  prosthesis  and 
prosthetic  applications.  This  approach  hierarchically  clusters 
the  work  distribution  among  the  levels  to  enable  high 
computational  speed  and  processing.  Three  PICs  (peripheral 
interface  controllers )  were  used:  one  32-bit  PIC  and  two  16- 
bit  PICs.  These  made  up  the  multi-level  hierarchical  model. 
Communication  between  the  levels  was  established  by  full- 
duplex  distributed  communication.  This  model  facilitates  to 
modify  the  communication  system  between  the  levels  as  for  the 
application.  An  intelligent  fuzzy  controller  associated  with  the 
higher-level  configuration  provides  a  control  strategy  for  the 
lower-level  configuration  to  control  the  end  device,  in  our 
case  the  stepper  motor.  The  stepper  motor  provides  desired 
step  changes  in  the  output  angular  displacement,  depending 
on  the  application.  A  rectified  sine  wave  input  was  fed  to  the 
controller,  which  pulled  up  a  set  of  rules  from  the  membership 
function  of  the  intelligent  fuzzy  controller  to  initiate  a  desired 
angular  displacement. 

Keywords  -  Fuzzy  Controller,  Multi-Level  Control,  Embedded 
Systems,  Full  Duplex  Distributed  Communication 


i.  Nomenclature 

The  following  acronyms  are  used  in  this  paper  to  represent 
the  motor  actuation  signals  associated  with  our  proposed 
control  strategy: 

NB  -  Negative  Big, 

NM  -  Negative  Minimum, 

PB  -  Positive  Big, 

PM  -  Positive  Minimum. 

ii.  Introduction 

The  work  by  Liu  et.  al  [  1]  summarized  the  current  tools 
used  for  the  design  and  implementation  of  embedded  systems, 
which  lacked  some  important  phases,  e.g.  supervision  logic, 
real-time  scheduling,  design  modeling  of  communication,  etc. 
However,  in  today’s  real  world  there  is  more  concentration  on 
the  design  and  simulation  tools.  Usually  for  hybrid  control 
systems,  there  will  be  sub -controllers  and  switching  logic.  The 
complete  controller  will  be  defined  with  a  set  of  equations. 
This  paper  attempts  to  address  design  and  implementation  of 
heterogeneous  hierarchical  control  systems.  In  order  to  capture 
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different  design  perspectives,  to  demonstrate  design  principles, 
various  software  tools  were  used. 

Self-configuring  robust  embedded  systems  seek  to  achieve 
graceful  degradation,  and  this  could  be  achieved  by  software 
reconfiguration.  To  accomplish  this,  the  system  must 
automatically  reconfigure  or  upgrade  itself  despite  any  node 
failure.  This  paper  gives  architecture  for  the  typical  distributed 
embedded  control  systems  to  employ  real-time,  reliable  data 
transmission  using  control  area  network  (CAN)  [2]. 

PIC 

PIC  is  for  Programmable  Interface  Controller.  Owing  to  its 
low  cost,  often  free  development  tools,  serial  programmable 
capability,  PICs  are  used  in  many  applications. 

PIC32  was  introduced  by  Microchip  Technology  Inc.,  a 
leading  vendor  of  microcontroller  and  analog  semiconductors. 
PIC  uses  Harvard  architecture.  In  Harvard  architecture,  it  has 
separate  code  space  and  data  space.  The  code  space  is  often 
implemented  by  EPROM,  ROM  or  Flash  ROM.  Addressable 
data  space  was  extended  through  banking.  A  program  counter 
is  also  integrated  into  the  data  space  and  writable.  In  PIC32, 
the  highest  execution  speed  is  80  MIPS,  one  instruction  will  be 
executed  per  one  clock  cycle.  Generally  512  K  bytes  of  flash 
memory  is  used.  It  can  be  programmed  by  using  the  Microchip 
MPLAB  C  Compiler.  In  our  design,  the  MPLAB  ICD2  (with 
USB  2.0  interface)  was  used. 

Stepper  Motor 

Stepper  motor  is  an  open-loop  controller.  It  is  an 
electric  motor  which  can  divide  the  full  rotation  into  number 
of  steps  depending  upon  the  application.  In  a  unipolar  stepper 
motor  there  are  two  windings  per  phase,  one  for  corresponding 
direction  of  the  magnetic  field.  As  the  magnetic  field  can  be 
changed  without  making  any  changes  in  the  current  flow 
direction,  it’s  easy  to  make  the  circuit  for  winding.  In  the 
design,  for  a  given  phase,  one  end  of  winding  will  be  kept  as 
common  giving  three  leads  per  phase.  In  order  to  activate  the 
driver  transistors,  stepper  motor  controllers  are  used.  They  are 
used  to  get  the  angular  displacements. 

in.  Background 

Recent  research  shows  a  conjunction  of  design 
strategies  characteristic  of  robot  control  [3].  A  suite  of 
continuous  processes  is  used  by  many  architectures.  This  uses 
local  low-level  control  for  particular  motions  [3].  Aboveboard, 


a  task-level  control  is  required  so  that  lower- level  processes 
can  be  assigned  to  the  particular  tasks  [3].  Therefore,  there  is  a 
requirement  for  a  higher-level  planned  layer  to  grant  a  planned 
vision  for  other  required  tasks  [3]. 

Robot  systems,  such  as  autonomous  mobile  robots,  are 
required  to  organize  their  resources  (sensors,  actuators,  and 
computation),  and  need  to  recover  elegantly  from  exceptions 
[10].  Some  of  the  previous  research  by  [4]  decomposed  the 
problem  into  different  levels.  In  [4]  a  number  of  levels  of 
competence  are  defined  for  an  autonomous  mobile  robot. 

The  approach  in  [5]  is  to  design  robot  systems  as  a 
collection  of  behaviours  that  is  independent,  action- generating 
entities.  Uncertainty  and  unpredictable  changes  are  handled 
well  in  [5],  but  how  it  scales  as  tasks  and  environments 
increase  in  diversity  is  hazy.  The  capabilities  in  [5]  are  related 
to  task-level  control.  These  capabilities  form  the  basis  of  the 
executive  layer  of  the  three-tiered  robot  control  architectures  in 
[6],  [7],  [8],  and  [9]. 

The  architectures  in  [6],  [7],  [8],  and  [9],  included  three 
layers:  planning  layer,  executive  layer,  and  behaviour  layer. 
The  behaviour  layer  interacts  with  the  physical  world  and  is 
responsible  for  real-time  control  by  controlling  actuators  and 
collecting  information  from  sensors.  How  to  realize  goals  and 
how  to  handle  goal  interactions  is  stated  by  the  planning  layer, 
whereas  the  executive  layer  facilitate  between  the  symbolic 
level  of  the  planner  and  the  continuous  level  of  the  behaviours. 
Evidences  in  [6,  11,  12,  13,  14,  15,  16]  show  that  hybrid 
architectures  are  the  most  recent  research  outcomes.  The 
combination  of  the  reactive  and  deliberative  control  is  the 
advantage  of  hybrid  architectures.  Hybrid  architecture  takes 
the  advantage  of  efficient  low-level  control  with  a  connection 
to  high-level  reasoning.  Most  of  the  time  the  connection 
between  the  two  levels  is  difficult .  Therefore,  special  attention 
must  be  given  to  designing  and  implementing  this  connection 
to  achieve  the  right  blend  of  reactivity  and  deliberation  [17]. 

The  need  to  manage  the  complexity  of  interactions  - 
both  interactions  between  the  system  and  its  environment,  and 
interactions  between  different  parts  of  the  system  is  one  of  the 
definite  issues  while  designing  a  robotic  system.  The  best  way 
to  deal  with  this  complexity  is  divide  it  in  smaller  parts  and 
levels  with  different  levels  of  abstractions  [17].  Therefore,  by 
using  modularity  within  a  given  structure  of  the  architecture  - 
the  complexity  of  the  overall  system  can  be  simplified  by 
ranking  it  into  smaller  modules  [17].  The  data  flow  style 
architectures  facilitates  this  by  giving  independence  to  the 
upstream  and  downstream  levels  of  the  system  [17]. 
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Our  objective  of  this  design  is  to  give  the  computational  load 
to  the  higher- lev  el  processer  so  that  the  lower- lev  el  processors 
can  perform  the  required  motor  function  tasks. 

IV.  Proposed  Design 

The  proposed  design  is  a  multi-level  hierarchical  model 
with  a  rectified  sine  wave  input.  A  flash  memory  interfaces 
with  the  high-level  configuration,  making  it  capable  of  holding 
the  input  functions  and  the  control  strategy.  The  flash  memory 
interface  is  done  by  a  Secure  Digital  (SD)  card  which  enables 
quick  data  retrieval  and  processing.  The  32-bit  PIC,  memory 
interface  and  the  Fuzzy  controller  constitutes  the  higher-level 
configuration.  This  level  controls  the  lower- level  configuration 
depending  on  input  function.  Two  16-bit  PICs  each  associated 
with  a  stepper  motor  makes  up  the  lower-level  configuration. 
Two  lower- level  PICs  are  programmed  to  control  the  stepper 
motors.  The  communication  system  in  this  design  is  unique. 
The  two  levels  are  communicated  through  a  full  duplex 
distributed  communication  protocol.  Such  a  communication 
system  enables  an  effective  coordination  between  the  levels  by 
distributing  the  traffic,  and  therefore  eliminates  delay  caused 
often  by  traffic  congestion.  This  design  provides  flexibility  in 
handling  with  a  wide  range  of  communication  systems,  which 
could  include  both  hard-wired  and  wireless  communication 
based  on  the  application.  The  end  product,  i.e.  stepper  motors, 
provides  the  necessary  step  changes  in  the  output  displacement 
based  on  the  control  signal,  a  function  of  input.  The  proposed 
design  is  shown  in  Figure  1 . 


Based  on  the  input  function,  the  fuzzy  controller  of  the 
high-level  configuration  has  its  control  signal,  to  which  the  32- 
bit  PIC  has  access  to  it  through  the  flash  memory.  The  32-bit 
PIC  sends  the  encrypted  data  via  the  fully  duplex  distributed 
communication  channel  to  accesses  the  programed  lower- level 
PICs.  The  encrypted  data  access  the  two  16-bit  PICs  to  control 
the  two  stepper  motors  for  the  desired  angular  displacement. 
The  encrypted  data  contains  all  the  information  of  the  control 
signal.  It  also  contains  the  information  regarding  the  desired 
step  changes  and  the  direction  of  them.  The  lower-level  PICs 
are  preprogramed  to  control  the  stepper  motors  with  necessary 
angular  displacement.  Synchronization  plays  an  important  role 
in  the  lower-level  configuration.  The  encrypted  data  and  the 
communication  system  make  the  synchronization  easier.  This 
type  of  communication  systems  avoids  traffic  delay  time.  The 
encrypted  data  also  has  the  information  about  the  direction  and 
angular  displacement  for  each  motor.  The  critical  part  of  the 
model  is  the  fuzzy  controller.  This  gives  the  control  signal  to 
the  high-level  controller,  which  encrypts  the  data  and  send  it  to 
the  lower- level  configuration. 

More  specifically  the  lower-level  configuration  involves 
two  16-bit  PICs  that  contain  the  code  to  control  respective 
stepper  motors.  When  a  message  is  received  from  the  higher- 
level,  it  will  state  which  motor  it  wants  to  activate  and  what 
angular  displacement  it  will  make.  One  or  both  of  the  PIC  16s 
will  take  its  message  and  extract  the  angular  displacement. 
Once  this  is  done,  the  PIC  will  relay  a  message  to  the  stepper 
motor  making  it  turn  to  the  desired  position.  Once  this  is  done 
the  PIC16  will  then  send  a  message  to  the  PIC32  that  lets  the 
PIC32  know  that  it  has  completed  its  task.  This  information 
will  be  fed  into  the  fuzzy  controller  that  will  continue  with 
processing  the  data. 

The  higher-level  configuration  will  include  a  PIC32.  The 
PIC32  will  have  a  SD  card  reader  attached  to  it.  Having  an 
SD  card  will  allow  large  files  to  be  read  into  the  PIC.  Another 
key  part  of  the  high-level  configuration  is  a  fuzzy  controller. 
The  fuzzy  controller  will  be  used  to  analyze  the  data  and 
process  it.  The  rectified  sine  wave  is  compared  to  the  set  of 
rules  specified  in  the  membership  function  of  the  intelligent 
controller.  Once  a  result  is  determined,  the  controller  will  send 
a  message  to  the  lower-level  controller.  The  message  will 
specify  which  PIC  16  will  be  getting  the  instruction  and  the 
desired  angular  displacement  of  the  stepper  motor. 


Figure  1:  Block  Diagram  of  the  Proposed  Design 
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Y.  Control  strategy 

Fuzzy  controller  works  on  a  fuzzy  logic.  Fuzzy  logic  can 
be  described  as  computing  with  words  rather  than  numbers. 
Therefore,  fuzzy  controller  can  be  defined  as  controller  with 
sentences  rather  than  equations.  The  main  components  of  the 
fuzzy  control  structure  include:  Fuzzification,  Rule  Base  and 
Inference  Engine,  and  Defuzzification,  as  in  Figure  2  [1,  18]. 

Fuzzy  controller 


Figure  2:  Fuzzy  Controller  System  Diagram 

Fuzzification  is  defined  as  taking  out  the  rules  from  rule 
base  given  an  input  function.  The  rule  base  is  a  set  of  rules, 
which  are  defined  by  the  user,  depending  upon  the  application. 
The  resulting  control  signal,  which  is  a  function  of  the  rule 
base  and  input,  is  called  Defuzzification. 


Figure  3:  Input  Signal  (Rectified  Sine  Wave) 


In  this  proposed  design,  the  rectified  sine  wave  is  fed  as  an 
input  signal  as  shown  in  figure  [2].  The  rectified  sine  wave  is 
sampled  at  regular  intervals  of  time  based  on  the  degrees  of 
freedom. 


Membership  Function: 

If  ‘x’  is  too  small  and  ‘y’  is  too  small  then  output  is  ‘NB’ 

If  4x’  is  small  and  ‘y’  is  small  then  output  is  ‘NM’ 

If  4x’  is  large  and  ‘y’  is  large  then  output  is  ‘PB’ 

If  V  is  too  large  and  ‘y’  small  then  output  is  ‘PM’ 

The  control  signals  are  fed  to  the  lower-level  configuration 
as  inputs.  The  lower-level  configuration  outputs  them  to  the 
end  user,  i.e.  the  stepper  motors  as  angular  displacements.  The 
control  signals  and  their  corresponding  angular  displacements 
are  shown  in  Table  1. 


Table  1:  Rules  for  Fuzzy  Controller 


Control  Signal 

Angular  Displacement 

NB 

Anti  clock  wise  90° 

NM 

Anti  clock  wise  45° 

PB 

Clock  wise  90° 

PM 

Clock  wise  45° 

Either  the  change  in  the  time  or  magnitude  pulls  up  a  rule 
in  the  Rule  Base  from  which  a  corresponding  fuzzy  controller 
signal  is  sent  to  the  PIC32.  For  this  design,  the  degree  of 
freedom  is  four.  Therefore  the  membership  function  consists 
of  four  rules.  Enhancements  of  degree  of  freedom  make  the 
membership  function  contain  more  rules.  Table  2  shows  the 
synchronization  chart  for  the  individual  control  signal  for  the 
lower-level  configuration.  This  synchronizes  the  two  stepper 
motors  for  the  application. 


Table  2:  Synchronization  Chart 


Control  Signal 

PIC1 

PIC2 

NB 

NB 

PB 

NM 

NM 

PM 

PB 

PB 

NB 

PM 

PM 

NM 

vi.  Experimental  Results  And  Discussion 

Figures  4  and  5  show  the  state  diagrams  for  the  higher-level 
and  lower-level  configurations.  Table  1  tabulates  the  control 
signal  for  the  lower-level  configuration  sent  in  the  form  of 
encrypted  data  by  the  higher-level.  This  encrypted  data  access 
the  preprogrammed  lower-level  configuration  for  a  necessary 
angular  displacement  by  the  stepper  motor.  Table  2  shows  the 
synchronization  chart  for  the  two  lower-level  PICs,  which 
between  the  two  PICs  can  be  done  by  a  unique  control  signal. 
The  chart  can  be  changed  based  on  individual  applications. 
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IVoccsvx  (PIC  -32) 


Figure  4:  State  Diagram  for  the  Processor  (PIC32) 


ConttuSci  (PIC  - 16) 


Figure  5:  State  Diagram  for  the  Controller  (PIC16) 


The  encrypted  data  access  the  program  in  the  lower- level 
configuration.  This  enables  the  lower-level  configuration  to 
make  necessary  angular  displacement  in  steps.  The  step 
change  and  the  direction  of  the  angular  rotation  is  function  of 
the  input.  So  finally,  this  model  becomes  an  independent  finite 
state  machine,  which  generates  a  directed  step  changes  in 
output  for  the  corresponding  input  function.  The  encrypted 
data  is  a  series  of  characters  communicated  through  a  full 
duplex  communication  system,  and  these  characters  alter  the 
direction  and  step  changes  in  the  output.  Controlling  the 
embedded  platform  through  hyper  terminal  also  involves 
control  of  the  end  device  through  characters.  This  permits 
serial  transmission  of  data.  The  same  concept  is  used  in  this 
configuration.  However,  using  hyper  terminal  to  control  an 


embedded  platform  for  prosthetic  application  is  not  efficient, 
because  the  platform  should  be  independent. 

For  programming  the  two-level  configuration,  MPLAB 
along  with  the  ICD2  (in  circuit  debugger)  is  used.  ICD2  helps 
in  programming  the  microcontrollers  while  in  the  circuit.  This 
enabled  us  to  make  changes  in  the  membership  functions  and 
the  synchronization  rules  quickly  according  to  the  user 
applications.  After  the  programming,  the  MPLAB  is  set  to 
release  mode  and  this  enables  the  two-level  configuration  to 
discard  from  the  computer  and  be  independent,  and  work 
independently  to  its  input  function.  For  programming,  CCS  C 
compilers  are  used. 


Table  3:  Experimental  Results 


Control 

Signal 

Motorl 

Angular 

Displacement 

Motor2 

Angular 

Displacement 

NB 

Counter-Clockwise  90° 

Clockwise  90° 

NM 

Counter-Clockwise  45° 

Clockwise  45° 

PB 

Clockwise  90° 

Counter-Clockwise  90° 

PM 

Clockwise  45° 

Clockwise  45° 

From  Table  3,  we  see  that  each  control  signal  contributes  a 
synchronized  angular  displacement  for  the  individual  stepper 
motors  in  either  the  clockwise  or  counter-clockwise  directions. 

vii.  Conclusions  And  Future  Work 

This  paper  proposed  a  multi-level  embedded  platform  for 
the  control  of  prosthesis.  In  this  model,  we  used  a  32-bit  PIC 
with  a  fuzzy  controller  and  memory  interface  as  a  higher- lev  el 
configuration  and  2  16-bit  PICs  for  lower-level  configuration. 
The  encrypted  data  from  the  higher- level  configuration  is  send 
to  the  lower-level  configuration.  This  is  transmitted  through  a 
full  duplex  distributed  communication  system.  This  increases 
the  transmission  speed  without  any  congestion.  Based  on  the 
encrypted  data,  lower-level  configuration  makes  the  stepper 
motors  generate  the  resulting  angular  displacement.  Therefore 
the  resulting  angular  displacement  of  the  stepper  motor  was  a 
function  of  the  input,  a  rectified  sine  wave  in  this  case. 

For  future  work,  it  will  be  interesting  to  implement  this 
design  with  even  more  powerful  processors.  This  will  improve 
the  computational  speed.  It  is  challenging  to  implement  this 
design  with  a  hybrid  fuzzy  controller  with  a  bio  signal,  e.g. 
EMG  (or  electromyogram).  The  hybrid  fuzzy  controller  should 
be  integrated  with  a  feedback  loop  for  further  improvements. 
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ABSTRACT 

Multi-sensor  data  fusion  has  an  incredible  impact  in  the 
field  of  wireless  sensor  networks  since  the  last  couple  of 
decades.  Many  theories  and  statistical  proofs  have  been  made 
to  enhance  the  design  and  system  in  view  of  accuracy  and 
reliability  in  particular.  Researchers  have  always  looked  into 
these  two  factors  by  modeling  and  computing  for  better 
accuracy,  according  to  user  application.  But  not  much 
emphasis  is  given  to  feedback  and  memory.  A  novel  approach 
for  a  sensor  network  that  provides  a  feedback  is  proposed, 
keeping  energy  conservation  in  mind.  A  look  into  some  possible 
energy  concerns  are  discussed,  visualizing  different  scenarios 
that  can  occur.  This  paper  proposes  a  distributed  de¬ 
centralized  feedback  system  with  energy  conservation  schemes, 
especially  for  data  fusion  networks  in  a  wireless  remote 
environment.  It  presents  an  innovative  node-based  model  in  a 
dataflow  chart,  introducing  feedback  into  the  wireless  world  of 
sensor  networks.  It  also  looks  into  various  scenarios  that  can 
cause  inaccuracy,  redundancy  and  other  glitches  in  certain 
events,  and  proposes  solutions  to  these  issues.  The  model  in 
this  paper  is  designed  for  a  broad  perspective  of  applications 
that  depend  on  data  fusion  for  accuracy.  If  accuracy  is  not 
satisfied,  a  feedback  is  produced  from  the  node  level  itself  to 
request  further  data  from  the  sensors. 

Keywords:  Sensor  Networks,  Data  Fusion,  Node  Selection, 
Multisensory  Feedback,  Wireless  Sensors 

INTRODUCTION 

Fusion  of  multisensory  data  provides  a  substantial  benefit 
over  a  single  source  of  data.  Information  from  additional  data 
sources,  when  fused,  delivers  precision  when  compared  to  a 
single  data  source  [11].  Though  originally  developed  for 
military  purposes,  multisensory  data  fusion  soon  took  its  place 
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in  many  other  evolving  applications.  This  paper  focuses  its 
contribution  in  wireless  sensor  networks. 

Multisensory  data  fusion  in  wireless  network  environments 
can  cover  a  broad  spectrum  of  applications  such  as  battlefield 
surveillance,  remote  sensing,  automatic  target  tracking  and 
detection,  [3].  This  paper  challenges  applications  related  to 
exploration.  Few  examples  that  can  be  thought  of  are  forest 
habitat  research,  underwater  exploration,  outer  space  discovery 
or  its  related  fields. 

This  paper  explains  the  concept  of  the  “node”  architecture. 
It  looks  into  important  blocks  in  the  process  flow,  explaining 
their  functions  in  the  process.  It  also  visualizes  scenarios  that 
could  take  place  in  the  basic  process  of  its  operation.  An 
illustration  of  the  node-to-node  snap  and  its  immediate  network 
recovery  is  presented.  As  limited  power  is  one  of  the  major 
concerns  in  the  wireless  network  environment,  a  look  into  the 
possible  power  related  issues  is  given.  Considering  the  power 
issues,  our  “node”  model  tries  to  diffuse  and  spread  the 
workload  over  the  entire  network.  This  provides  an  energy 
aware  routing  scheme  for  the  system.  Thus,  the  entire  sensor 
network  is  balanced  with  respect  to  traffic  load,  and  longevity 
of  the  system  is  enhanced. 

PROPOSED  NODE-BASED  ARCHITECTURE 

Our  proposed  node  architecture  consists  of  a  Sensor 
module,  a  Data  Fusion  module,  a  Fetch  Trigger,  a  Knowledge 
Base  and  Reference  module,  and  the  Network.  The  Sensor 
module  is  the  main  source  of  information  to  the  entire  network. 
It  consists  of  a  sensor  (raw  data  measurement)  and  an  event 
interrupt  device,  which  self-triggers  the  node  for  any  changes  in 
the  environment.  The  Data  Fusion  module  provides  two 
processes:  data  fusion  and  data  aggregation.  These  two 
processes  work  on  energy  threshold  levels  that  decide  if  the 
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node  should  perform  data  fusion  or  data  aggregation.  Data 
aggregation  is  considered  a  subset  of  data  fusion.  This  process 
is  only  chosen  if  it  falls  under  a  certain  threshold  level.  If 
energy  levels  are  above  this  threshold  level,  data  fusion 
(including  data  aggregation)  will  be  performed.  The  working  of 
the  entire  system  is  explained  in  latter  sections  of  this  paper. 
The  architecture  for  our  model  is  depicted  in  Figure  1 . 


Fig.  1:  Node-based  architecture  showing  all  node  modules 


This  architecture  introduces  a  feedback  to  the  node-based 
model  through  a  module  called  the  Fetch  Trigger.  A  Fetch 
signal  is  only  triggered  (to  its  lower  nodes)  if  the  measured 
values  do  not  meet  the  required  accuracy  level.  The  Knowledge 
Base  and  Reference  module  is  regarded  as  the  chief  central 
command,  and  provides  reference  for  the  node.  It  checks  if  the 
node  has  to  operate  as  a  sensor,  or  fuse  data,  or  check  for 
interrupts,  or  simply  relay  data  into  the  network.  The  network 
cloud  decides  which  node  to  route,  according  to  the  routing 
protocols  programmed  (i.e.,  application  specific).  It  also 
decides  if  the  signal  needs  to  be  modulated  or  demodulated  into 
a  Data  signal  or  a  Fetch  signal.  All  these  are  explained  as 
below. 


PROPOSED  NODE-BASED  DATA  FLOW 

As  technology  enhances,  it  is  becoming  more  possible  for 
faster  processors  and  large  sized  memory  to  work  with  low 
power  consumption.  Consider  each  of  the  sensor  nodes  built 
with  these  features,  thus  allowing  a  faster  throughput  for  a  data 
fusion  setup.  As  we  look  into  the  proposed  node  model,  the 
structure  of  this  network  system  supports  a  de-centralized, 
distributed  architecture.  This  node  structure  is  mainly  designed 


to  keep  work  distributed  between  nodes,  and  spread  them 
throughout  the  network.  We  look  into  an  environment  where  all 
these  sensor  nodes  are  spread  out  in  an  ad  hoc  manner. 
Assuming  each  node  in  the  network  has  a  unique  identity,  same 
functionality,  and  consists  of  a  built-in  timer  already 
synchronized  before  dispatch.  We  now  examine  the  model  as 
the  user  initiates  the  network. 

The  Sink  sends  out  the  Fetch  signal  to  all  the  nodes,  with 
the  request  of  the  location  address  to  visualize  the  network. 
Each  node  then  passes  the  Fetch  data  till  the  end  node  is 
reached.  The  final  node  basically  plays  the  role  of  the  sensor, 
which  is  to  measure  data.  Then,  the  reverse  path  takes  place 
from  the  source  sensor  back  to  the  Sink.  This  measured  data  is 
then  sent  to  the  Network  module.  It  is  the  Network  that  decides 
which  path  and  direction  to  take,  and  which  node  to  transfer  its 
information.  Figure  2  shows  the  data  flow  diagram  of  our 
proposed  model. 

It  should  be  observed  that  there  are  two  different  signal 
types  that  run  through  each  sensor  node.  The  Fetch  and  Data 
are  the  two  signals  used  in  this  paper  to  differentiate  the 
direction  of  flow.  The  Fetch  signal  is  only  used  to  find  the  end 
sensor  node  in  the  network.  This  Fetch  signal  does  not  carry 
any  data,  but  is  rather  used  to  request  lower  nodes  to  reply  back 
with  data.  Hence,  Fetch  can  be  modulated  as  a  low  power 
signal  with  lesser  bandwidth  range.  This  signal  keeps  going 
through  the  network  until  it  reaches  its  end  node.  Once  the  end 
node  is  detected  (as  the  end  node  will  have  no  possible  lower 
branch  nodes),  the  Fetch  signal  is  converted  into  a  Data  signal 
through  the  Modulator/D  e-modulator  unit  of  the  network  cloud. 
From  Figure  2,  at  the  end  node  check  block,  if  the 
Modulator/De-modulator  recognizes  the  signal  as  Fetch,  then 
the  system  will  operate  in  search  for  the  end  node.  And  if 
Modulator/De-modulator  recognizes  the  signal  as  a  Data  signal, 
it  will  search  for  the  Sink,  as  it  moves  in  the  opposite  direction. 

After  the  end  sensor  nodes  sense  information,  they  send  raw 
measured  data  to  their  respective  immediate  parent  nodes. 
These  nodes  check  with  their  lower  nodes  for  measured  values. 
If  there  are  no  other  nodes  carrying  the  same  data  type,  then  its 
current  value  and  location  address  is  stored  and  sent  to  the 
Network  module  to  be  transmitted  to  the  next  node.  If  there  is 
more  than  one  node  having  measured  values,  then  it  is  sent  to  a 
checker  for  data  fusion.  This  is  explained  later  in  this  paper.  If 
Data  Fusion  (depicted  as  DF  in  the  flow  chart)  was  not  able  to 
take  place  because  of  node  energy  constraints,  then  Data 
Aggregation  takes  place.  Data  Aggregation  will  only  collect 
data  from  its  lower  nodes  and  pass  it  onto  the  next  node  which 
gets  routed  by  the  Network  module. 


Page  364  of  531 


I  Mod /De-Mod  I 

:  r ; 

I  Tx./RxH 


Fig.  2:  Data  flow  for  the  nodal  feedback  model 


respective  branched  parents.  Considering  this  dynamic 
environment,  where  A  and  D  are  two  end  nodes.  Node  E  keeps 
drifting  away  from  its  child  node  D,  then  a  network  snap 
occurs. 


Fig.  3:  An  example  of  a  network  snap 


Here  now,  A  becomes  the  adopted  parent  for  D,  and  E 
becomes  the  end  sensor.  E  will  sense  information  and  send  it 
through  F  while  D  sends  its  measured  information  through  A. 
The  advantage  for  all  nodes  equipped  with  the  same 
functionality  promotes  reliability  of  the  entire  system. 
Anywhere  down  its  data  flow  path,  if  any  failure  occurs,  the 
system  can  stabilize  and  retrieve  information  back  to  the 
network.  This  is  a  benefit  for  this  kind  of  network  models  in  the 
dynamic  environment. 


We  introduce  an  event  interrupt  into  the  system  as  a 
software  program.  It  is  part  of  the  Sensor  module,  and  is 
programmed  to  trigger  a  self-initiated  request  to  the  Sink.  Once 
the  sensor  detects  a  change  in  the  environment  worthy  to  report, 
an  interrupt  gets  generated.  After  the  Sink  knows  how  its 
network  structure  is  distributed,  it  can  individually  send  a  Fetch 
request  to  a  particular  node  when  needed.  This  process  of  node 
selection  is  enhanced  from  its  traditional  algorithm,  using  the 
relevant  node  selection  algorithm  [10].  This  algorithm  claims 
energy  efficient  querying.  Thus  overall,  there  are  three  ways 
this  network  communicates  -  Sink-all  sources-Sink ,  Sink-single 
source-Sink ,  and  self -interrupt  driven  source-Sink. 

We  look  into  a  scenario  where  two  end  sensor  nodes  are  in 
close  proximity  with  each  other  (shown  in  Figure  3).  Then  a 
misperception  occurs  as  to  which  node  should  act  as  a  sensor. 
In  this  case,  both  must  act  as  Sensors  and  reply  back  to  its 


POWER  CHECK  FOR  DATA  FUSION 

Power  in  wireless  sensor  networks  is  one  of  the  major 
concerns,  as  they  run  remotely  on  just  battery.  Introducing  data 
fusion  on  these  systems  will  call  for  more  processing  power. 
One  way  to  manage  power  levels  for  each  node  is  by 
introducing  power  threshold  levels  into  each  node.  Targeting 
data  fusion  applications,  we  introduce  two  threshold  levels,  T1 
and  T2,  into  the  power  check  scheme  for  each  node. 


Below  T2  >  T2  >  T1 

(Data  transfer)  (Data  Aggregation)  (Data  Fusion) 
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Power  check  resulting  over  threshold  level  T1  is  considered 
for  data  fusion  of  previous  nodes  (if  any).  Power  levels  less 
than  Tl,  or  equal  to  Tl,  will  call  for  data  to  be  aggregated  or 
collected.  Power  levels  less  than  or  equal  to  T2  will  only  send 
raw,  unprocessed  data  over  to  the  next  node.  These  threshold 
levels  can  be  set  after  testing  the  node  for  power  performance. 
This  block,  as  depicted  in  Figure  2,  compares  its  own  health 
with  reported  health  of  its  neighbors  [4] . 

Apart  from  the  Data  Fusion  process,  other  areas  that  cause  a 
lot  of  power  consumption  are  data  storage  in  memory  [9]  and 
signal  transmission  between  nodes  [4] .  Typically,  there  are  two 
ways  to  store  the  collected  data:  one  is  to  store  them  in  the 
network  nodes;  the  other  is  to  store  them  in  the  Sink  node  [9]. 
This  paper  supports  data  storage  in  all  the  network  nodes  for  its 
significance  in  data  fusion.  Likewise,  when  a  particular  node  is 
selected  (assuming  any  node  in  the  network  but  the  end  nodes) 
to  send  data  back  to  the  Sink,  it  will  already  hold  previously 
fused  data,  and  can  compare  it  with  the  currently  measured 
data.  With  this,  accurate  values  can  be  derived  from  only  one 
data  source. 

It  should  be  noted  that  data  fusion  will  not  always  take 
place.  Chances  are,  most  of  the  data  fusion  that  take  place  will 
be  at  the  Sink  node,  or  the  user,  where  power  is  not  a  limitation. 
This  occurs  when  only  one  node  is  called  to  process 
information  and  send  back  the  data  through  the  rest  of  the  relay 
nodes.  The  other  possibility  is  when  two  or  more  nodes  are 
called  to  retrieve  data,  and  they  follow  separate  node  paths. 
Data  Fusion  occurs  during  power  up  of  the  system,  where  the 
Sink  wants  to  know  the  distributed  location  of  the  network 
nodes.  Furthermore,  it  occurs  when  multiple  signals  (Data  or 
Fetch)  intersect  at  a  common  node.  Power  consumption  can 
also  be  minimized  by  communicating  only  when  necessary. 
Some  of  these  basic  power  awareness  schemes  should  be 
considered  to  drastically  improve  node  power  conservation  [1]. 

KNOWLEDGEBASE  AND  REFERENCE 

This  module  controls  all  processing  of  the  node  and  keeps 
refreshing  the  knowledgebase.  The  importance  of  this  unit  is  to 
update  the  respective  node  with  recent  information,  location  of 
its  neighboring  nodes  and  hop  counts.  This  will  update  each 
successive  parent  node  respectively,  as  Data  signals  are  called 
back  by  the  Sink.  It  becomes  evident  that  higher  order  nodes 
(towards  Sink)  will  have  an  updated  Knowledge  Base,  because 
of  already  fused  data  from  different  paths.  This  in  turn  forces 
lower  order  nodes  to  perform  data  fusion  at  earlier  stages. 
Emphasis  is  given  on  lower  order  node  data  fusion  so  that 
higher  order  node  energy  levels  can  be  maintained.  Higher 
order  nodes  mostly  get  a  knowledgebase  update  and  check  for 
further  data  fusion.  The  higher  the  node’s  order,  the  greater  its 


priority,  as  it  holds  a  string  of  network  nodes.  They  are  also 
valued  because  of  their  proximity  to  the  Sink.  It  balances  load 
at  each  node  and  a  smoother  traffic  is  obtained.  A  Data  Fusion 
power  level  unit  checks  to  see  if  the  node  has  enough  energy  to 
perform  data  fusion.  If  energy  is  insufficient  then  it  simply 
passes  the  data  fusion  task  on  to  the  next  node. 

During  initial  stages  or  right  after  the  dispatch  of  the 
network  nodes,  data  fusion  would  take  place  at  much  lower 
node  levels  as  power  is  assumed  to  be  maximal.  This  would 
still  not  cause  early  power  drain  at  lower  node  levels. 
Considering  the  first  iteration  that  caused  data  fusion  at  lower 
nodes.  The  next  iteration  would  only  send  raw  data  back  to 
parent  nodes  as  it  would  be  at  T2  (threshold  level  2)  or  below. 
So  the  power  drain  is  also  watched  at  the  lower  level  nodes. 
Overall,  the  lower  levels  keep  pushing  more  work  toward  the 
higher  levels  as  time  progresses.  This  not  only  balances  the 
work  load  of  the  network  but  also  provides  timeliness  (fast 
enough  to  use)  by  distributing  small  processes  per  node  over 
the  network.  Also,  if  the  nodes  are  dispatched  in  the  hundreds 
or  thousands,  an  accurate,  balanced  traffic  load  and  long  lasting 
network  system  is  created  preliminarily. 

Studies  show  that  a  central  control  or  a  centralized  network 
where  all  the  collected  data  is  fused  at  a  single  processor  helps 
to  avoid  the  possibility  of  duplication  [5].  However  on  the 
proposed  de-centralized  network,  this  should  not  be  a  concern, 
as  each  node  is  updated  with  the  source  address  and  trajectory 
node  addresses.  The  node  that  actually  performs  the  data  fusion 
will  check  for  redundant  node  addresses  and  simply  disregard 
them  if  any  such  addresses  exist. 

The  Knowledge  Base  and  References  module  already  has  a 
set  of  predefined  function  values.  It  is  application  specific,  as 
moisture  sensors  would  have  a  different  reference  than  light 
sensors.  Another  important  function  of  the  Knowledge  Base  is 
to  keep  updating  itself  on  the  location  addresses  or  previous 
nodes.  This  helps  the  network  to  recognize  the  dynamic  change 
in  the  system.  It  also  keeps  updating  itself  with  the  hop  count 
from  each  node,  which  is  essential  to  calculate  the  power  levels 
so  as  to  minimize  the  transmission  cost  [4] . 

MODULATOR  /  DE-MODULATOR 

Considering  that  each  node  is  able  to  transmit  two  types  of 
signals,  namely  Data  signals  and  Fetch  signals.  The  assignment 
for  this  module  is  to  modulate  signals  based  on  the  direction  of 
its  travel.  If  a  Fetch  signal  is  sent  from  the  Sink,  then  all  the 
nodes  will  only  search  for  the  final  node,  and  that  node  will 
only  act  as  the  sensor.  On  its  return  path  back  to  the  Sink,  the 
modulator  then  changes  the  frequency  of  the  return  signal  and 
calls  it  the  Data  signal.  Only  Data  signals  can  be  processed  for 
Data  Fusion.  The  introduction  of  the  modulator/de- modulator 
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provides  the  security  for  the  system.  The  Mod/De-Mod  can  be 
designed  according  to  the  application  requirements. 

The  Mod/De- Mod  promotes  its  signal  over  the  transmitter/ 
receiver  unit.  The  Mod/De- Mod  unit  converts  the  signal  into 
three  instances,  namely: 

•  End  node  conversion:  Once  the  Fetch  signal 
reaches  the  end  node,  this  node  converts  the  Fetch 
signal  into  a  Data  signal  (i.e.  coming  back). 

•  Event  interrupt:  If  a  sensor  detects  a  change  to 
report,  this  unit  requests  a  Data  signal  conversion 
at  the  Mod/De-mod. 

•  Feedback  for  inaccurate  nodes:  Checks  for 
accuracy  criteria  in  lower  order  nodes  and  triggers 
inaccurate  nodes  by  generating  Fetch  signal  at  the 
Mod/De-Mod  unit. 


FEEDBACK 

This  paper  introduces  a  feedback  system  into  data  fusion, 
for  the  wireless  sensor  network  environment.  As  we  have 
assumed  that  each  node  is  equally  equipped  with  the  same 
functionality,  each  node  should  be  able  to  trigger  a  feedback  to 
its  lower  level  node  or  nodes.  This  feedback  is  only  generated, 
based  on  the  decision  criteria  of  the  data  fusion  algorithm 
implemented  as  per  the  application.  The  feedback  loop  consists 
of  three  blocks  that  check  to  see  if  a  feedback  is  required.  The 
Data  Fusion  module  checks  sensed  data  for  accuracy  or  any 
requirement  with  respect  to  the  Knowledgebase.  If  any  node 
disagrees  with  the  sensed  data  before  fusing,  it  sends  out  a 
Fetch  to  its  lower  nodes  for  a  reconfirmation.  Then  it  compares 
it  with  the  previous  values  of  the  Knowledgebase,  and  passes  a 
Data  signal  on  backward  to  the  higher  order  node.  For  a  clearer 
understanding,  we  look  at  an  example  considering  three  nodes 
A,  B,  C,  where  C  is  the  parent  node  for  both  A  and  B  (Figure 
4).  Considering  a  decision  criterion  based  on  a  probability  data 
fusion  model,  where  A  provides  70%  accuracy  and  B  provides 
only  50%  accuracy  (for  a  knowledgebase  accuracy  of  80%). 
Since  both  the  expected  values  are  below  the  accuracy 
requirement,  a  feedback  will  be  triggered.  But  if  the  accuracy 
level  is  say  60%,  then  C  only  triggers  B.  This  is  again  based  on 
the  Knowledgebase  requirements  to  set  the  accuracy  percentage 
in  this  case.  If  the  accuracy  from  both  A  and  B  are  met  with  the 
Knowledgebase  of  C,  data  fusion  will  be  performed. 


Fig.  4:  Feedback  in  two  scenarios:  (a)  Both  A,  B  are 
inaccurate  with  C,  (b)  Only  B  is  inaccurate  with  C 


It  can  be  thought  that  the  power  of  the  overall  system  may 
drain,  especially  for  higher  order  node  levels.  As  they  are  peers 
for  lower  order  nodes,  sending  Fetch  from  this  point  all  the  way 
to  end  nodes  can  dramatically  drain  the  power,  especially  for  a 
large  number  of  iterations.  But  this  should  not  be  a  concern  as 
feedback  should  take  place  only  at  the  lower  order.  This  is 
because  at  lower  levels  the  accuracy  requirement  is  already 
met.  Otherwise  the  data  fusion  would  not  take  place.  This  is  the 
main  rationale  behind  our  proposed  nodal  feedback  model. 

DATAFUSION  AND  NETWORK  ROUTING 

Data  fusion  is  preferred  and  is  a  valued  process  because  of 
its  accuracy  in  complex  scenarios  and  ability  to  work  in  the 
wireless  sensor  network  [2].  It  has  already  succeeded  in  many 
applications  in  the  wireless  area  [2]  [3].  There  are  many  types  of 
data  fusion  models,  proposed  and  enhanced  since  last  few 
decades  [2]  [5].  Some  models  work  well  for  a  certain  type  of 
applications,  while  other  models  work  better  for  some  other 
applications.  While,  one  fusion  model  looks  into  Active 
perception  (SeFMAP)  for  Human  Identification  [7],  other 
fusion  model  looks  into  Bayesian  approaches  for  Prosthetic 
hands  [8].  Subsequently,  the  Data  Fusion  block,  the  Network 
Routing  block,  and  the  Knowledgebase  block  are  application 
specific.  These  are  left  open  for  flexibility,  according  to  the 
application’s  design  and  specific  requirements. 

The  Network  Routing  block  processes  information  about 
the  node  address,  identification,  and  itinerary  or  each  node,  as 
discussed  previously.  This  paper  looked  into  adaptive  and 
probabilistic  routing  algorithms,  because  of  each  node  being 
updated  of  node  address,  hop  count,  etc.  [6].  But,  this  block  is 
also  left  open  to  programmers  for  best  routing  schemes. 
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SYNCHRONIZATION 

The  synchronization  of  this  network  is  interrupt  driven. 
Each  node  is  triggered  only  when:  (a)  Sink  calls  a  particular 
node,  (b)  when  the  Sensor  module  detects  an  interrupt,  and  (c)  a 
feedback  is  generated  due  to  lack  of  accuracy  in  a  node.  Apart 
from  this,  synchronization  of  nodes  takes  place  when  the  entire 
sensor  network  is  powered  up  or  when  a  data  fusion  is 
expected.  As  our  objective  does  not  include  synchronization 
techniques,  it  is  left  for  implementers  to  choose  the  appropriate 
synchronization  algorithm  to  suit  this  nodal  model. 

CONCLUSION  AND  FUTURE  WORK 

In  this  paper  we  presented  a  feedback  based  nodal  model 
for  wireless  sensor  networks.  We  proposed  an  architecture 
design  in  the  viewpoint  of  each  node  and  its  contributions  to 
successfully  fuse  data  along  its  reverse  path  to  the  sink.  We 
discussed  some  power  issues  and  provided  solutions.  Our 
model  also  emphasized  the  distribution  of  workload  over  the 
network,  so  as  to  balance  and  preserve  the  longevity  of  the 
overall  system.  Security  can  be  provided  to  the  system  by 
modulating  and  demodulating  Fetch  signal  and  Data  signal. 
These  signals  give  the  direction  of  flow  and  provide  the 
itinerary  for  the  next  node. 

A  node-based  data  processing  flow,  in  the  form  of  a  flow 
chart,  was  proposed  and  explained.  The  concept  of  how  the 
feedback  works  in  these  systems  is  illustrated  with  examples. 
The  persistence  to  keep  our  model  generalized  was  to  challenge 
new  application  approaches  that  may  suit  this  concept.  Future 
work  on  this  concept  is  to  produce  a  prototype  or  simulation  of 
this  model. 
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Abstract  -  In  this  paper  we  developed  a  framework  for  a 
computer  vision  system  to  determine  kinematic  information  of 
the  human  hand  motion,  i.e.,  joint  motion  in  discrete  intervals 
of  time.  Since  a  2D  computer  vision  system  developed  does  not 
provide  complete  kinematic  information  of  the  human  hand,  to 
overcome  this  we  need  a  3D  vision  system  that  uses  multiple 
cameras  and  reconstructs  a  3D  image  using  reconstruction 
techniques.  The  resulting  3D  image  is  used  to  determine  the 
joint  angles  for  all  joints  in  the  human  hand.  The  vision  system 
obtained  is  relatively  inexpensive  compared  to  commercial 
applications  available,  and  can  be  used  by  persons  who  do  not 
have  expert  knowledge  of  vision  systems. 

Keywords:  Computer  Vision,  3D  Reconstruction  (3D-RCon), 
Hand  Tracking  Analysis,  and  Motion  Capture. 

1  Introduction 

In  recent  years,  image  processing  and  human  hand  motion 
tracking  have  seen  an  increased  interest  in  human- computer 
interaction  (HCI)  related  applications.  Hand  motion  is  mainly 
defined  by  movement  of  underlying  skeleton.  This  skeleton 
can  be  modelled  as  an  articulated  structure  of  rigid  links 
connected  at  rotational  joints  forming  a  kinematic  chain.  The 
information  from  a  vision  system  based  on  this  kinematics  can 
work  with  electromyogram  (EMG)  signals  from  the  forearm 
muscles  to  provide  better  insight  into  how  the  hand  moves. 

Suppose  when  images  are  captured  depth  information  of 
scene  is  lost,  we  need  a  3D  vision  system  to  find  this  depth 
information.  According  to  Hong  Zhang  et.  al  [1],  for  a  3D 
vision  based  hand  motion  tracking  system  to  be  successful,  it 
has  to  be  widely  available,  simple,  accurate,  and  inexpensive. 


The  system  should  be  fast,  since  most  human  hand  tracking 
applications  are  real-time  in  performance  ;  and  should  be  able 
to  deal  with  occlusions  that  occur  while  computation  . 

This  paper  focuses  on  the  design  and  evaluation  of  a  vision 
system  for  hand  motion  tracking  and  capturing.  We  propose  a 
vision  system  that  captures  multiple  view-point  images,  which 
provides  three-dimensional  data  in  real  time.  To  elaborate  on 
this  work,  Section  2  reviews  the  background  research  in  3D 
imaging  and  motion  tracking  methods,  such  as  hand  pose 
estimation  for  HCI.  Section  3  describes  our  3D  vision  system 
design.  Section  4  discusses  the  experimental  input  and  results 
for  our  six-camera  system,  followed  by  Section  5,  conclusion 
and  future  work.  Acknowledgement  and  References  are  given 
in  Sections  6  and  7,  respectively. 

2  Background 

In  a  3D  vision  system,  imaging  and  motion  tracking  of 
human  hand  are  important  research  problems.  Their  various 
applications  such  as  human  body,  hands,  and  face  animation, 
pose  gesture  analysis  and  automatic  detection  of  human 
activities,  and  other  information  such  as  articulated  objects 
modeling,  are  often  necessary.  Vision  based  systems  can  be 
categorized  as  appearance-based  and  model-based.  Example: 
the  main  idea  behind  appearance-based  approach  is  detecting 
the  shape  of  the  hand.  Hoshino  and  Tanimoto  [2]  identified 
the  hand  posture  by  shape  detection  and  searching  it  from  a 
database  of  already  stored  hand  postures,  which  construct  a 
3D  model  of  the  hand  to  obtain  all  positional  and  kinematic 
information  of  hand.  But  in  this  case  it  is  a  tedious  procedure 
since  hand  is  a  complex  object  with  a  high  number  of  degrees 
of  freedom.  Vlasic,  Baran,  Matusiky,  and  Jovan  Popovi'c  [3] 
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developed  a  practical  software  system  for  capturing  mesh 
animation  details  from  multi-view-point  video  recordings, 
which  captures  the  motion  of  both  the  skeleton  and  the  shape 
and  the  result  of  this  output  mesh  animation  is  enhanced  with 
the  details  observed  in  the  image  silhouettes.  But  this  method 
cannot  reproduce  the  surface  accurately  from  the  contours, 
since  it  has  to  rely  on  the  template  to  interpolate  geometric 
information.  This  is  especially  problematic  for  unarticulated 
objects  such  as  long  scarves,  faces,  or  flowing  hair,  it  is  also 
sensitive  to  errors  in  the  silhouettes,  and  will  likely  produce 
incorrect  geometry  when  visual  hulls  are  noisy.  In  [4],  Ueda, 
Matsumoto,  Imai,  and  Ogasawara  proposed  a  novel  method 
for  hand  pose  estimation  that  can  be  used  as  input  interface. 
The  system  estimates  all  joint  angles  to  manipulate  objects  in 
the  virtual  space.  In  that  method,  a  hand  pose  is  reconstructed 
as  a  “voxel  model”  by  a  multi- view  point  camera  system,  then 
all  joint  angles  are  estimated  using  3D  model  fitting  between 
the  hand  model  and  voxel  model.  But  their  system  used  an 
estimation  that  caused  errors  in  the  hand  model.  Processing 
speed  of  system  was  slow.  The  research  in  [5]  by  Duraisamy, 
Perez,  and  Schoen  proposed  that  each  finger  is  described  as  a 
serial  robot  expressed  as  Clifford  algebra  exponentials.  This 
allows  us  to  perform  kinematic  synthesis  to  obtain  the  angles 
at  each  joint,  and  to  adapt  the  model  to  the  dimensions  of  the 
hand  of  the  subject  being  captured,  based  on  the  visual  data 
obtained  from  the  motion  tracking  system.  The  kinematic 
model  obtained  quantifies  the  motion  of  the  natural  hand.  The 
experimental  data  obtained  from  the  3  cameras  has  been  used 
to  obtain  some  simplified  results  for  the  index  finger.  In  our 
approach,  we  are  using  6  cameras  and  for  synthesizing  model 
for  whole  hand  for  any  arbitrary  motion.  However,  all  above 
mentioned  approaches  have  their  specific  concerns. 

In  our  research,  we  are  continuing  the  work  performed  by 
Hidayatullah  Ahsan  [6].  His  approach  was  to  develop  a  2D 
vision  system  with  a  marker-based  semi-automatic  tracking 
system  to  provide  the  joint  angles  of  the  index  finger,  which 
under  certain  assumptions  can  be  used  to  study  the  complex 
grasping  motion.  Though  effective,  this  approach  does  not 
provide  complete  kinematic  information  of  the  human  hand. 
To  overcome  this,  we  developed  a  3D  vision  system. 


algebra  seems  to  produce  a  very  good-behaved  system  of 
equations  for  the  numerical  solution.  The  equations  are  solved 
hierarchically  using  a  Levenberg-Marquardt  solver.  The 
preliminary  results  of  kinematics  for  smart  prosthetics  show 
that  it  is  possible  to  recover  the  hand  geometry  and  motion 
quickly  and  accurately  when  using  synthetic  data. 

3.1  System  Requirements 

Our  vision  motion  tracking  system  runs  on  a  workstation 
using  Intel®  Core™  2  Quad  CPU  at  2.40  GHz,  and  uses  6 
DragonFly®2  cameras,  model  DR2-BW/COL-XX  by  point 
Grey  Research  Inc.,  which  can  capture  black  and  white  and 
color  images.  This  system  is  connected  to  the  computer  with 
6-pin  IEEE1394a  interface.  These  cameras  can  be  used  with 
National  Instruments  (NI)  Lab  VIEW  software  [8]  [9].  For  a 
multiple-camera  vision  system  such  as  this  one,  it  is  important 
to  have  these  cameras  synchronized  with  a  maximal  deviation 
of  125ps  between  the  cameras  [9].  Fujinon  HF9HA-1B  9mm 
model  lens  are  used  for  our  system  due  to  its  affordability  and 
availability.  Accuracy  of  3D  object  reconstruction  depends  on 
camera  configuration,  lens,  resolution,  and  speed  [7].  Figure  1 
gives  the  camera  component  specifications  of  our  system. 


Point  Grey  Research  Inc. 


Model 

DR2-BW/COL-XX 

Sensor 

1/3"  Progressive  Scan  CCD 

Sensor  Pixel  Size 

7.4pm  Square  Pixel 

Color  Modes 

Black  and  White  (Mono  8  and  Mono  16) 

Color  (RGB,  YUV) 

Resolution 

Up  to  648x488 

Frame  Rate 

Black  and  White  (Up  to  60  fps) 

Color  (Up  to  30  fps) 

Model 

HF9HAlB9mm 

Mount  Type 

C/CS-  Mount 

Imager  Size 

2/3"  CCD 

Aperture 

f/1.4to  f/1.6 

Iris 

Manual 

Focus 

Manual 

Lens  by  Fujinon 


Figure  1:  Camera  and  lens  specifications 

3.2  Overview  of  the  System 


3  3D  Vision  System 

A  3D  vision  system  concerns  the  science  and  technology  of 
machines.  As  a  scientific  discipline,  computer  vision  is 
concerned  with  the  theory  behind  artificial  systems  that 
extract  information  from  images.  The  image  data  can  take 
many  forms,  such  as  video  sequences,  views  from  multiple 
cameras,  or  multi-dimensional  data.  The  hand  model  in  [5] 
consists  of  links  and  joints  that  have  specified  locations  and 
orientations  drawn  from  anatomic  data.  These  joint  locations 
and  orientations  can  be  adapted  to  the  actual  dimensions  of 
the  subject  via  the  kinematic  synthesis.  In  order  to  both  adapt 
the  hand  model  to  the  geometry  of  the  captured  hand,  and  to 
track  the  angle  at  each  joint,  the  procedure  follows  that  of 
real-time  model-based  human  motion  tracking  and  analysis 
for  human  computer  interaction  systems.  Applying  kinematic 
synthesis  expressing  the  motion  as  elements  of  Clifford 


Our  system  exploits  several  computer  vision  techniques  and 
multiple  camera  views  of  hand  to  find  the  depth  information. 
It  is  important  to  know  about  the  scene's  background.  This 
information  helps  in  extracting  the  foreground  object  from  an 
image.  For  our  computer  vision  system  we  use  a  solid  blue 
color  sheet  so  the  background  is  smooth  and  distinguishable 
from  the  human  hand.  For  a  multiple  camera  vision  system, 
the  cameras  can  be  placed  randomly  or  at  a  uniform  distance 
from  each  other,  as  shown  in  Figure  3.  The  cameras  should  be 
pointing  to  the  same  region  in  the  three-dimensional  space. 
The  process  of  3D  scene  reconstruction  depends  on  accurate 
camera  calibration  and  background  subtraction  [7],  and  can  be 
divided  into  the  following  steps: 

1 .  Camera  calibration 

2.  Image  acquisition 

3.  Background  subtraction 

4.  3D  object  reconstruction 
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Figure  2:  Steps  involved  in  3D  reconstruction 

3.3  Details  of  the  3D  Reconstruction 


with  an  input  of  sequence  images  i  =  1,  2...  ,  N.  The  SACON 
method  involves  the  following  steps: 

A.  Sampling:  In  this  step  the  absolute  difference  between 
the  pixel  intensities  of  image  i  and  all  background  images  t  = 
1. . .  ,  N  is  used  to  determine  if  a  pixel  m  in  image  i  is  part  of 
the  background  or  the  object  of  interest.  Such  an  observation 
can  be  expressed  by  formula  3.1: 

(l  if  |x,  (m)  -  xs  (m)  |  <Tr  or  f]  <  <  y 

Ai-|  xr(m) 

v  0  otherwise  (3.1) 

where  x,  (m)  represents  the  intensity  of  the  pixel  location  m  in 
image  i.  Here  Tr  is  the  error  tolerance  that  needs  to  be  found 
experimentally.  The  ratio  xt  (m)  /  xt  (m)  is  used  to  remove  the 
influence  of  shadow  in  an  image. 

B.  Consensus:  The  information  from  the  above  step  is  used 
to  create  a  mask  by  summing  the  sampling  results  of  an  image 
i  with  all  background  images  as  shown  in  formula  3.2: 

*,&»)=  (>  YjXm)-T- 

v  0  otherwise  n  21 


The  details  of  the  3D  reconstruction  are  described  below. 
Figure  2  illustrates  the  steps  involved: 

Camera  calibration:  The  first  step  for  3D  reconstruction 
using  multiple  images  is  to  find  a  viewpoint  for  each  camera, 
which  results  in  a  projection  model  [10]  that  is  first  used  to 
determine  the  viewpoint  of  each  image  and  then  to  find  the 
approximate  volume  in  space  that  contains  the  3D  object  [11]. 
This  gives  the  intrinsic/extrinsic  parameters.  For  calibrating 
multiple  cameras  simultaneously,  the  method  described  in 
[10]  was  used,  which  relies  on  correspondence  of  a  set  of 
optical  markers  between  images,  where  the  optical  marker  can 
be  a  source  of  light  such  as  a  laser  pointer. 

Image  acquisition:  As  discussed  in  Section  2,  for  image 
capturing  we  used  the  DragonFly®2  cameras  and  Lab  VIEW 
8.2  package  developed  for  capturing  multiple  view  of  hand 
motion.  We  captured  gray-scale  images,  since  the  frame  rate 
is  much  slower  for  color  images. 


Figure  3:  Camera  setup  for  image  acquisition 

Background  subtraction:  We  used  the  SACON  method 
[12]  developed  by  Wang  and  Sutter  for  segmenting  images  for 
tracking  applications  in  cluttered  or  dynamic  background, 


where  Tn  is  a  threshold  for  the  data  points  that  are  within  the 
error  tolerance.  Tn  is  a  function  of  Tr  and  can  be  found  using 
formula  3.3: 

Tn  =  tTtN  (3.3) 

where  x  is  a  constant  found  experimentally. 

C.  Visual  hull:  Silhouettes  for  all  images  at  a  particular 
instance  in  time  will  be  created,  which  have  the  projection 
transformation  of  the  camera.  Then  they  can  be  used  to  back 
project  the  foreground  region  into  the  3D  space  [13],  which 
results  in  a  conic  volume  that  contains  object  of  interest  visual 
hull.  The  term  “visual  hull”  was  coined  by  Laurentini  in  [14]. 
For  our  system,  we  constructed  a  cube  containing  the  object  of 
interest  and  the  dimensions  of  cube  found  by  trial  and  error. 
The  main  objective  is  to  find  the  point  in  cloud  cube  that  is 
part  of  the  3D  object.  This  problem  was  solved  by  using  an 
image  space  rather  than  an  object  space  [15]. 

Figure  5  is  the  expected  result  of  the  visual  hull  of  hand 
using  the  grid  point  cube  shown  in  Figure  4.  We  projected 
onto  the  silhouette  of  each  image  in  the  region  of  interest  or 
foreground  in  silhouette  image.  Otherwise  the  image  is 
discarded.  This  process,  when  applied  to  all  points  in  the 
bounding  volume  and  all  silhouettes,  results  in  a  visual  hull. 


Figures  4,  5:  Partitioned  cube  data  of  visual  hull  and  hand 
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D.  Surface  extraction:  The  visual  hull  that  has  point  cloud 
of  objects  contains  all  possible  points.  In  our  approach  the 
point  cloud  is  sliced  to  xy-plane  of  the  3D  coordinate  system. 
For  each  slice  the  interior  points  are  removed  based  on  simple 
neighborhood  method. 

E.  Surface  fitting:  The  surface  point  cloud  of  a  3D  object 
may  not  show  any  abnormalities  in  the  3D  object.  We  have 
found  that  the  best  way  to  observe  the  3D  object  abnormalities 
is  by  looking  at  the  surface  that  best  fits  the  surface  point 
cloud.  For  our  research  we  used  an  application  called  Tight 
Cocone  [16].  This  application  finds  a  water-tight  surface  to  a 
given  point  cloud  [17]. 

4  Experimental  Input  and  Results 

We  have  worked  on  several  image-processing  techniques  to 
capture  the  hand  motion.  Figure  6  presents  a  sequence  of  input 
images  of  human  hand  from  six  different  camera  views  that 
give  details  of  the  human  hand.  It  is  important  to  have  the 
information  of  scene  background,  since  it  helps  in  extracting 
the  foreground.  Alignment  of  the  hand  is  different  at  different 
intervals  of  time.  This  sequence  of  images  is  the  initial  input 
to  our  3D  system. 


Side  view  1  Side  view  2  Top  view  3 


Side  view  4  Side  view  5  Side  view  6 

Figure  6:  Captured  images  -  camera  views  1, 2,  3, 4,  5,  6 


captured.  The  joint  angles  of  the  fingers  are  computed  using 
formula  4.1.  This  is  an  off-line  post- processing  method. 

/  (x2-xi)(*3-*2)+(y2-yi)(y3-y2)  \ 

6  =  arccos  — 

V ((*2  -  *l)2  +  (>’2  ~  }’l)2)((*3  -*2)2+(>'3-  y’lY))  (4.1) 

where  (*2,y2)  end  (x3,y3)  E  h2  are  the 

coordinates  of  the  markers. 

4.2  Background  Subtraction  Results 

Figure  7  is  the  result  of  background  subtraction  in  dynamic 
background.  We  needed  to  remove  the  background  of  the 
scene  from  these  images  and  create  a  binary  image  with  pixel 
values  0  for  background  and  1  for  the  object  of  interest.  This 
resulted  in  silhouettes  representing  the  accurate  boundary  of 
the  object.  We  noticed  some  abnormalities  in  these  results 
since  image  background  is  different  for  each  camera. 


Figure  7:  Background  subtraction  results  with  six  cameras 


4.1  Hand  Motion  Tracking 

Hand  motion  tracking  is  used  in  hand  pose  estimation.  For 
our  research,  the  main  goal  is  to  determine  the  location  of  the 
finger  joints.  For  this  we  developed  a  3D  vision  setup  which 
includes  six  cameras  as  well  as  markers  drawn  on  finger  joints 
using  ink  pens.  The  input  to  this  system  is  a  sequence  of  gray¬ 
scale  images  captured  from  the  cameras.  When  the  image 
sequence  of  hand  motion  is  captured  (using  Lab  VIEW),  these 
markers  must  be  visible  in  all  cameras.  Simple  threshholding 
operation  is  performed  so  that  only  marker  images  are  left, 
which  store  the  3D  marker  position  information  at  particular 
intervals  of  time.  Further,  the  distance  between  these  marker 
positions  is  calculated  using  Euclidean  distance  between  two 
points.  For  this  marker-based  method  to  work  properly,  the 
markers  must  not  be  affected  by  external  lighting  conditions, 
orientation  and  camera  position,  under  which  images  are 


4.3  Discussion  of  Preliminary  Results 

Figure  8  shows  the  results  of  the  automatic  calibration 
method,  which  yields  a  complete  camera  projection.  These 
projections  of  the  light  source  shown  in  the  form  of  circles  are 
detected  independently  for  each  camera.  Every  time  when  the 
calibration  is  done,  it  is  important  to  know  whether  the  data  is 
valid.  If  needed,  we  can  always  recalibrate  the  system 
completely.  However,  this  takes  time,  and  the  resulting 
parameters  will  not  be  exactly  the  same  as  the  old  ones,  even 
though  the  setup  remained  the  same.  This  is  a  somewhat  trial- 
and-error  process.  We  need  to  strive  for  the  results  with  the 
lowest  reprojection  error  and  variance. 
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Figure  8:  Reprojected  outliers  (RANSAC  for  6  cameras) 

Figure  9  shows  the  results  of  reconstructed  setup  of  our  six- 
camera  system.  Small  blue  circles  with  numbers  denote  the 
identification  numbers  and  positions  of  the  camera  centers. 
Blue  lines  denote  orientation  of  the  cameras’  optical  axes.  The 
red  circles  are  the  reconstructed  positions  of  the  laser  pointer. 
From  this  we  observe  that  camera  alignment  is  not  yet  proper, 
the  reason  for  this  alignment  being  lens  distortion.  For  the 
vision  system,  all  cameras  need  to  be  closer  to  the  hand  so  that 
most  of  the  image  is  occupied  by  the  hand.  This  is  necessary 
to  get  detailed  images  of  the  hand  with  the  working  distance 
of  the  Fujinon  lens. 


Figure  9:  Reconstructed  axes  of  the  six-camera  setup 
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Figure  10  shows  the  average  reprojection  errors  and  their 
standard  deviations  for  each  camera  within  the  system.  While 
the  standard  deviations  are  deemed  acceptable,  the  mean 
errors  are  considered  high.  The  purpose  of  system  calibration 
is  to  minimize  both  the  mean  error  and  the  standard  deviation. 


2D  error:  mean  (blue),  std  (red) 


1  2  3  4  5  6 

Id  of  the  camera 


Figure  10:  Reprojection  errors  of  the  cameras 


5  Conclusion  and  Future  Work 

As  we  mentioned,  this  work  is  an  extension  of  a  previous 
system,  where  we  used  the  techniques  for  3D  reconstruction 
and  motion-tracking  marker-based  system,  in  which  various 
image-analyzing  techniques  were  applied  to  obtain  the  joint 
angle  data.  But  there  were  several  abnormalities  in  the  3D 
hand  that  was  reconstructed.  For  example,  two  thumbs  are 
visible  and  some  fingers  were  altogether  (this  can  be  referred 
in  [6]).  These  abnormalities  exist  because  we  are  using  only 
four  camera  views  and  background  for  all  images  is  dynamic 
and  it  varies  in  instant  interval  of  time.  So,  in  our  proposed 
system  we  introduced  a  six-camera  design  which  can  improve 
the  system  accuracy  and  reduce  abnormalities  by  placing 
cameras  as  close  as  possible  to  the  object,  i.e.  the  hand. 

For  future  work  we  plan  to  design  and  develop  a  real-time, 
fully  automatic  GUI-based  3D  computer  vision  system  that 
will  capture  the  images  in  a  dynamic  background  and  give  a 
better  insight  into  the  angle  data.  We  will  also  be  investigating 
any  potential  issues  related  to  scaling  up  of  the  system. 
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Abstract 

Genetic  Algorithm  (GA)  are  biologically  motivated 
evolutionary  computation  techniques  used  both  as 
search  methods  for  solving  problems  and  for 
modelling  evolutionary  sy stems. GA  has  been  used  in  a 
wide  variety  of  optimization  tasks,  including  numerical 
optimization  and  combinatorial  optimization  problems 
such  as  circuit  design  and  job  shop  scheduling. 
Although  there  are  many  problems  for  which  the 
genetic  algorithm  can  evolve  into  a  good  solution  in 
reasonable  time,  there  are  also  problems  for  which  it 
is  inappropriate  like  the  one  in  which  it  is  important  to 
find  the  exact  global  optimum.  In  addition,  if  the 
number  of  iteration  increases  the  algorithm  will  take  a 
long  time  to  complete  its  computations.  In  this  work, 
we  are  using  GA  to  optimize  the  parameters  of 
nonlinear  Bayesian  Filters  for  electromyographic 
(EMG)  signal.  OpenMP  is  a  parallel  programming 
model  for  shared  memory  and  distributed  shared 
memory  multiprocessors.  We  are  using  OpenMP  on 
four  processors  to  improve  the  time  of  G A  optimization 
for  Bayesian  Filter  parameters. 

Keywords:  OpenMP,  Matlab g ,  MPI,  EMG,  GA, 
Gene,  Mother  Chromosome,  Father  Chromosome, 
Pairing  Operator,  Mating  Operator,  Mutation, 
Population  Size. 

1.  Introduction 

Principles  of  natural  selection  and  genetics  has  been 
used  to  deduce  some  efficient  search  algorithms,  GA  is 
one  of  them.  GA  is  used  in  different  fields  to  obtain 
solutions  to  different  problems,  e.g.  engineering, 


science,  business  and  economics  [1,  2].  Some  of  the 
applications  of  GA  are  in  automotive  design, 
engineering  design,  robotics,  evolvable  hardware, 
optimized  telecommunication  routing,  joke  and  pun 
generation,  bio-mimetic  invention,  traffic  and 
shipment  routing,  computer  gaming,  encryption  and 
code  breaking,  computer-aided  molecular  design,  gene 
expression  profiling,  optimizing  chemical  kinetic 
analysis,  finance  and  investment  strategies,  and 
marketing  and  merchandising.  Although,  for  a  general 
problem  of  normal  size,  GA  is  a  good  option  to  get  the 
optimal  solution  in  appropriate  amount  of  time, 
whereas  for  bigger  and  complex  problems  GA  takes 
more  time  to  find  optimal  solution.  Researchers  have 
explored  different  approaches  to  minimize  this 
computational  time  and  cost  and  parallel  processing 
came  out  to  be  one  of  the  most  promising  approaches. 

GA  is  based  on  the  principle  of  natural  selection.  It 
uses  a  random  search  technique  to  find  the  optimal 
solution  of  a  problem  by  manipulating  a  population  of 
candidate  solutions.  To  form  next  generation,  GA 
selects  the  best  solutions  from  the  evaluated 
population.  Good  solutions  and  time  to  reach  them  by 
GA  depends  on  the  population  size  [3,  4],  because  if 
the  population  is  too  small,  then  the  search  space  will 
not  be  sufficient  explored  and  it  will  be  difficult  to 
identify  good  solutions.  In  the  case  when  the 
population  is  too  big,  the  GA  will  waste  computational 
resources  processing  unnecessary  individuals.  Each 
individual  in  the  population  has  a  fitness  value,  which 
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is  a  payoff  measure  that  depends  on  how  well  the 
individual  solves  the  problem.  Crossover  and  mutation 
are  the  two  operators  used  by  GA  to  explore  the  search 
space,  here  crossover  is  a  primary  and  mutation  is  a 
secondary  search  operator.  The  probability  of 
crossover  is  much  higher  than  the  probability  of 
mutation.  GA  can  be  stopped  based  on  different 
conditions,  one  is  by  using  a  predetermined  number  of 
generations  and  function  evaluations,  second  is  the 
average  quality  of  the  population  gets  saturated  after 
some  number  of  generations,  and  the  third  is  when  all 
the  individuals  are  identical,  which  can  only  occur 
when  mutation  is  not  used.  We  are  using 
predetermined  number  of  generations  and  function 
evaluations  to  stop  the  GA  in  this  research  work.  The 
generic  GA  flow  chart  is  shown  in  Figure  1 . 


Figure  1 :  Generic  GA  Flow  Chart. 


The  time  taken  by  GA  is  directly  proportional  to  the 
number  of  evaluations  and  generations,  the  more  the 
number  of  evaluations  the  lager  is  the  computational 
time  for  GA.  On  the  other  hand,  this  time  can  be 
minimized  by  paralleling  the  GA  program  on  a  parallel 
cluster.  Most  of  the  parallel  programs  divide  the  task  in 
chunks  and  solve  it  simultaneously  using  multiple 
processors.  On  a  parallel  cluster,  there  are  numerous 
processors,  and  they  share  the  task,  the  larger  the 
number  of  processors  the  faster  the  computation  or 
lesser  the  time  for  the  completion  of  the  algorithm.  In 
this  research  work  we  are  using  OpenMP  parallel 
programming  model  and  utilizing  a  cluster  of  four 
processors  to  run  the  GA  and  comparing  its 
performance  with  the  single  processor  in  \fatlab%  . 


2.  Background 

Most  of  the  parallel  programs  divide  the  task  into 
chunks  and  solve  it  simultaneously  using  multiple 
processors.  Similar  approach  can  be  used  for  GAs  in 
different  ways.  These  approaches  can  be  using  a  single 
population,  divide  the  population  into  several 
subpopulations,  use  of  massively  parallel  architecture, 
or  use  of  fewer  multicomputer  with  fewer  and  more 
powerful  processing  units.  Mainly  there  are  three  kinds 
of  parallel  GAs,  and  these  are  (a)  global  single¬ 
population  master- slave  GAs,  (b)  single-population 
fine  grained,  and  (c)  multiple-population  coarse¬ 
grained  GAs.  The  hierarchical  parallel  GAs  combine 
the  multiple  demes  with  master-slave  or  fine-grained 
GAs.  It  has  a  higher  level  and  a  lower  level,  at  the 
higher  level  they  are  multiple-deme  algorithms  and  at 
the  lower  level  they  are  single-population  parallel  GAs 
(either  master-slave  or  fine-grained)  [1]. 

The  research  work  in  [5]  shows  a  method  of  parallel 
GA,  in  which  a  global  parallel  implementation  of  the 
conventional  GA  and  a  GA  with  a  generation  gap, 
where  some  part  of  the  population  is  replaced.  The 
efficiency  of  global  parallel  GA  is  close  to  100  %  in 
SIMD  (single  instruction  multiple  data)  computers  by 

[5] .  Four  prototypes  for  parallel  GAs  are  proposed  in 

[6]  where  the  first  three  are  variations  of  the  master- 
slave  GA,  and  the  fourth  is  a  multiple-population  GA. 

2.1  Master-Slave  Parallelization 

Same  as  in  a  simple  GA,  master-slave  GA  also  have 
a  single  panmictic  population.  Figure  2  shows  the 
schematic  of  master- slave  GA,  where  the  evaluation  of 
fitness  is  distributed  among  the  slave  processors. 


Figure  2:  Master- Slave  Parallel  GA. 
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The  master-slave  GA  considers  full  population  size 
for  selection  and  crossover,  so  it  is  known  as  global 
GA.  The  master-slave  parallelization  method  for  GA 
does  not  change  the  behavior  of  the  algorithm  while 
the  single-population  fine  grained  and  multiple- 
population  coarse-grained  GA  change  the  way  the  GA 
works  [1].  The  master-slave  parallel  GA  considers  all 
the  population  for  selection,  whereas  the  other  parallel 
GAs  considers  only  a  subset  of  the  population.  In 
addition,  in  the  master-slave  parallel  GA  the  mating  is 
random,  whereas  in  the  other  methods  mating  is 
restricted  to  a  subset  of  individuals  [1].  The  majority  of 
the  global  parallel  GAs  are  implemented 
synchronously.  If  the  multiprocessor  architecture  is  of 
shared  memory  then  each  individual  processor  can 
share  the  information,  reading  and  writing  without  any 
issue.  In  contrast,  on  distributed-memory  architecture, 
the  population  is  stored  only  at  one  processor  and  the 
master  is  responsible  for  sending  the  individuals  to 
slaves  for  evaluation,  collecting  the  results  and 
applying  GA  for  next  generation.  Encore  Multimax  is 
a  shared-memory  computer  with  16  processors,  a  GA 
is  implemented  on  this  to  search  for  efficient 
timetables  for  schools  in  [7].  This  research  in  [7]  gave 
a  limited  speed-up  because  of  the  critical  path  of  the 
program  serial  code. 

In  1993,  Abramson,  Mills,  and  Perkins  added  a 
distributed-memory  machine  “Fujitsu  API 000”  with 
128  processors  to  the  experiments,  changed  the 
application  to  train  timetables  and  modified  the  code 
[8].  In  this  research,  the  speed-up  was  significant  up  to 
16  processors  and  then  decreased  because  of  the 
increase  in  communication  as  more  processors  are 
added. 

Hauser  and  Manner  used  global  GA  on  three 
different  parallel  processors  and  got  good  speed  up 
only  on  NERV  multiprocessor  [9].  The  speed-up  of 
five  using  six  processors  is  obtained  with  a  very  low 
communications  overhead  [9].  The  reason  of  low 
performance  on  other  processors  a  SparcServer  and  a 
KSR1  is  because  of  the  inadequate  scheduling  of 
computation  threads  to  processors  by  the  system.  All 
the  previous  research  results  shows  that  the  master- 
slave  parallel  GAs  are  easy  to  implement  and  very 
efficient  method  of  parallelization  especially  when  the 
evaluation  needs  considerable  computations. 
Additionally  the  master- slave  parallel  GAs  do  not 
change  the  search  behavior  of  the  GA  [1]. 


3.  GA  In  Matlab  - 

The  continuous  number  GA,  which  is  used  in  this 
work,  interprets  the  parameters  of  a  problem  as  genes, 
where  one  set  of  parameters  form  a  chromosome, 
chromosome  —  [gene1,  gene2,  — , geneNpCir} .  Since 

one  chromosome  corresponds  to  a  candidate  solution, 
its  evaluation  is  done  by  assessing  the  numerical  value 
of  that  solution,  which  usually  referred  as  the  cost. 
Unlike  traditional  optimization,  the  function  is  not 
referred  to  as  the  cost  function  but  as  the  fitness 
function  or  objective  function.  Using  an  initial 
population  size  with  randomly  generated 
chromosomes,  where  each  gene  is  generated  by  a 
random  number  generator  within  the  search  space,  the 
chromosome  will  take  the  dimension  of 

RNpar*populatum  sue  After  evaluating  each 

chromosome  cost 

=  / (chromosome j),  i  =  1,2, ... ,  population  size , 

the  selection  operator  will  select  a  subset  of  the 
chromosomes  based  on  their  cost  value  to  survive  for 
the  next  iteration. 

For  elitism  based  GA,  the  best  performing 
chromosomes  always  survive.  There  are  two  options 
the  population  size,  a  steady- state  or  a  performance 
based  size.  The  first  option  maintains  a  certain  number 
of  chromosomes  for  each  iteration  -  also  referred  to  as 
a  generation;  while  performance  based  size,  selection 
sets  a  minimum  criterion  for  chromosomes  to  survive. 
The  latter  option  may  cause  the  algorithm  to  end 
prematurely  since  not  minimum  criteria  (cost  value). 

Pairing  Operator:  Genetic  Algorithm  may  employ 
a  number  of  pairing  mechanisms.  Among  them  are 
pairing  from  top  to  bottom,  where  the  best  performing 
chromosome  is  paired  with  the  next  best  performing 
chromosome  and  so  on;  random  pairing,  where  a 
uniform  distribution  is  assumed  to  randomly  pick  the 
mother  and  father  chromosome;  weighted  random 
pairing  where  the  probability  of  selection  is  influenced 
by  the  performance  of  the  individual  chromosome;  and 
rank  weighing,  where  the  probability  of  selection  is 
influenced  by  the  ranking  of  chromosome  with  respect 
to  the  others  measured  again  by  the  performance.  In 
this  paper,  the  pairing  is  accomplished  by  determining 
the  cumulative  probability  for  each  chromosome  is 

poo  =  ~,  where 

^n=l 1 

_  population  size-no^)/  chromosomes  kevt 
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and  n  is  the  index  of  chromosomes.  Having  computed 
the  individual  cumulative  probabilities  P(n),  the 
parent  chromosomes  are  now  selected  by  utilizing  a 
random  number  generator  and  indexing  this  with  the 
cumulative  probability.  Hence,  the  best  performing 
chromosome  has  the  best  chance  of  being  selected,  but 
also,  all  chromosomes  in  the  mating  pool  have  a 
chance  of  being  selected. 

Mating  Operator:  In  this  work,  the  next  generation 
chromosomes  are  created  from  the  mother  and  father 
chromosome  (which  were  selected  as  described  above) 
by  following  equations  given  as: 


9e7W  neWi  = 

mother  (a)  —  (3  (mother  (tx)  —  father  (a)), 


gene  ^2  = 

father  (a)  +  ft  (mother  (a)  —  father  (a)), 

Where,  a  =  integer  (rand  *  iVpcr), 

/ 3  =  random  number  between  0  —  1,  and  Npar  is 
number  of  parameters  for  complete  solution.  In  this  a 
can  also  be  viewed  as  equivalent  to  the  cross-over 
point  utilized  in  the  binary  GA.  Hence,  the  offspring 
(new  chromosomes)  are  found  as 

chromneWi  - 

(ge™  mot  her t  . 9 enemother2 . gene  neiYi  •  "•* 

and 

chromreW2  =  (gene  iaii,  gene  d  ad2 . genereW2 

9enemBtkerN^). 

Mutation:  Mutation  is  an  important  operator  that 
allows  the  search  to  escape  local  minima  in  order  to 
find  better  areas  on  cost  surface.  Too  much  mutation 
will  do  opposite,  i.e.  if  a  chromosome  approach  an 
optimum  point,  it  might  mutate  and  be  placed  at  an 
entirely  different  location  of  the  cost  surface.  Mutation 
is  accomplished  by  randomly  changing  a  gene,  i.e. 
substitute  it  with  a  random  numbers. 


group,  which  represents  the  major  vendors  of  high- 
performance  computing  hardware  and  software.  A  set 
of  compiler  directives  is  the  major  part  of  OpenMP 
interface.  The  programmer  adds  these  compiler 
directives  to  a  sequential  program.  The  programmer 
specifies  the  synchronization  points  and  tells  the 
compiler  what  parts  of  the  program  to  execute 
concurrently.  As  the  directives  can  be  added  in  a 
incremental  fashion,  the  OpenMP  gives  a  path  and 
platform  for  the  parallelization  of  existing  software.  In 
contrary,  the  Pthreads  and  MPI  approaches  are  library 
routines.  Pthreads  and  MPI  are  linked  with  and  called 
from  a  sequential  program,  and  the  programmer  is 
required  to  divide  the  computational  work  manually 
[11]. 

Literature  does  not  show  any  evidence  of  the  use  of 
OpenMP  parallel  programming  model  for  GA  in 
specific.  This  work  is  using  OpenMP  to  optimize  the 
filter  parameters  of  Bayesian  filters  using  GA 
technique.  The  parallel  cluster  has  four  processors.  The 
idea  of  the  research  is  to  implement  the  GA  using 
OpenMP  on  this  four  processors  cluster  and  compare 
the  results  with  the  output  on  Matlab ®  on  a  single 
processor.  The  OpenMP  parallel  programming  model 
is  given  in  Figure  3,  where  sequential  parts  and  parallel 
regions  are  labelled  clearly. 


Figure  3 :  OpenMP  Parallel  Programming  Model. 

5.  Results  and  Discussions 


4.  GA  In  OpenMP 

OpenMP  is  a  parallel  programming  model,  which 
uses  some  compiler  directives  and  library  routines  to 
express  shared-memory  parallelism  [10].  OpenMP 
uses  Fortran  and  C++  interfaces,  which  is  a  standard 
feature.  OpenMP  API  is  designed  and  developed  by  a 


In  this  work,  we  are  using  GA  to  optimize  the 
parameters  of  nonlinear  Bayesian  Filters  for 
electromyographic  (EMG)  signals.  OpenMP  is  a 
parallel  programming  model  for  shared  memory  and 
distributed  shared  memory  multiprocessors.  We  are 
using  OpenMP  on  four  processors  to  improve  the  time 
of  GA  optimization  for  Bayesian  Filter  parameters. 
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First  Matlab 8  is  used  to  run  GA  on  one  processor, 
and  the  same  algorithm  is  applied  on  OpenMP  parallel 
processing  model  using  four  processors.  Two  sets  of 
data  captured  at  a  sampling  rate  of  2000Hz  are  used, 
first  data  is  for  30  seconds  and  the  second  is  for  60 
seconds.  The  total  simulation  time  taken  by  the 
Matlab ®  and  OpenMP  is  recorded  precisely. 
Following  tables  shows  the  results  obtained  from 
simulations.  Table  1,  2  and  3  shows  the  Matlab ® 
simulation  results  for  two  different  data  sets  of 
different  lengths. 


Table  1:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab ®  . 


No.  of 

Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 

set. 

Simulation  time 
seconds  -  60 
seconds  of  data 

set. 

10,000 

7.13 

2393.06 

15,000 

16.95 

2403.52 

20,000 

28.05 

2416.66 

25,000 

42.66 

2432.89 

30,000 

60.61 

2452.34 

35,000 

82.41 

2475.94 

40,000 

108.21 

2503.69 

45,000 

139.35 

2535.91 

50,000 

175.45 

2571.72 

Table  2:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab ®  . 


No.  of 
Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 

set. 

Simulation  time 
seconds  -  60 
seconds  of  data 

set. 

60,000 

208.65 

2613.84 

70,000 

260.67 

2741.73 

80,000 

331.85 

2813.34 

90,000 

428.03 

2913.25 

100,000 

550.89 

3039.70 

Table  3:  Simulation  time  for  30  seconds  and  60 
seconds  data  sets  in  Matlab ®  . 


No.  of 
Iterations 

Simulation  time 
seconds  -  30 
seconds  of  data 

set. 

Simulation  time 
seconds  -  60 
seconds  of  data 

set. 

125,000 

704.54 

3193.36 

150,000 

943.89 

3433.06 

175,000 

1288.28 

3775.16 

200,000 

1755.77 

4236.28 

Figures  1,  2  and  3  are  graphical  representation  of  the 
results  from  the  Table  1,  2  and  3  respectively.  It  is 
evident  from  Figure  1,  2  and  3  that  CPU  time  is 
directly  proportional  to  the  data  size  and  number  of 
iterations.  Therefore,  increase  in  the  data  size  and  the 


number  of  iterations  results  in  the  increase  in  CPU 
time. 


1.5  2  25  3  3.5  4  4.5  5 


No,  Of  Iterations  x10 

Figure  1 :  Iterations  Vs.  CPU  Time  -  Table  1  Data. 


6  6.5  7  7.5  8  8.5  3  9.5  10 

No.  Of  Iterations  ^ 

Figure  2:  Iterations  Vs.  CPU  Time  -  Table  2  Data. 


If  1.3  1.4  1.5  1.6  1.7  1.8  1.9  2 

No.  Of  Iterations  x,°s 

Figure  3:  Iterations  Vs.  CPU  Time  -  Table  3  Data. 
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The  GA  code  from  Matlab ®  is  converted  into  C- 
code  and  OpenMP  implementation  is  under  pipeline. 
Results  will  be  published  in  next  reputed  conferences. 
The  code  is  available  at  http://www.isu.edu/~chiustev/. 

6.  Summary  and  Future  Work 

To  conclude  this  work  uses  the  idea  of  parallel 
processing  and  implement  GA  on  OpenMP  parallel 
programming  model  of  four  processors.  First  Matlab* 
GA  code  is  converted  to  C  code  and  then  to  OpenMP 
to  implement  on  the  parallel  cluster.  The  algorithm  is 
implemented  both  in  OpenMP  and  Matlab ®  for  25, 
50,  75  and  100  iterations  and  the  time  taken  in 
execution  is  recorded.  As  expected  the  results  shows 
the  improvement  of  time  in  OpenMP  implementation. 
In  future  work,  similar  approach  will  be  applied  using 
MPI  and  OpenMP  and  MPI  combined  for  GA  and 
more  complex  and  advanced  optimization  algorithms. 
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The  dimensional  synthesis  of  spatial  chains  for  a  prescribed 
set  of  positions  can  be  applied  to  the  design  of  parallel  robots 
by  joining  the  solutions  of  each  serial  chain  at  the  end  ef¬ 
fector.  This  design  method  does  not  provide  with  the  knowl¬ 
edge  about  the  trajectory  between  task  positions  and,  in  some 
cases,  may  yield  a  system  with  negative  mobility.  These  prob¬ 
lems  can  be  avoided  for  some  overconstrained  but  movable 
linkages  if  the  finite  screw  system  associated  to  the  motion 
of  the  linkage  is  known.  The  finite  screw  system  defining  the 
motion  of  the  robot  is  generated  by  a  set  of  screws,  which 
can  be  related  to  the  set  of  finite  tasks  positions  traditionally 
used  in  the  synthesis  theory.  The  interest  of  this  paper  lies 
in  presenting  a  method  to  define  the  whole  workspace  of  the 
linkage  as  the  input  task  for  the  exact  dimensional  synthesis 
problem.  This  method  is  applied  to  the  spatial  RPRP  closed 
linkage,  for  which  one  solution  exists. 

Nomenclature 

S  Line  or  dual  vector,  defined  using  Plucker  coordinates, 
s  Vector 

S  Quaternion  or  dual  quaternion 
w  Dual  number 
[M\  Matrix 

1  Introduction 

Synthesis  of  parallel  robots  has  focused  mainly  on  type 
or  structural  synthesis,  using  group  theory,  screw  theory,  or 
geometric  methods,  see  for  instance  [1],  [2],  [3].  Dimen¬ 
sional  synthesis  examples  exist,  mainly  for  optimizing  per¬ 
formance  indices  [4],  [5],  [6]  or  for  reachable  workspace  siz¬ 
ing  [7],  [8],  [9];  see  also  [10]  for  a  comprehensive  approach. 

The  dimensional  synthesis  of  spatial  serial  chains  for 
a  prescribed  set  of  positions  can  be  used  for  the  design  of 
parallel  robots  by  synthesizing  all  supporting  legs  fo^the 
same  set  of  positions.  There  are  a  few  examples  of  fin?t%- 
position  dimensional  synthesis  of  parallel  robots  in  the  lit¬ 


erature,  most  of  them  doing  partial  synthesis.  Wolbrecht  et 
al.  [1 1]  perform  synthesis  of  3-RRS,  4-RRS  and  5-RRS  sym¬ 
metric  parallel  manipulators;  Kim  and  Tsai  [12]  and  Rao  [13] 
solve  the  partial  kinematic  synthesis  of  a  3-RPS  parallel  ma¬ 
nipulator.  This  method  has  been  successfully  applied  mainly 
to  special  parallel  systems  with  imposed  symmetry.  In  gen¬ 
eral,  the  method  does  not  allow  the  control  of  the  final  trajec¬ 
tory  of  the  parallel  system;  in  the  most  extreme  cases,  it  may 
yield  a  system  with  negative  mobility,  that  can  be  assembled 
at  each  task  positions  but  cannot  be  driven  from  task  position 
to  task  position. 

The  kinematic  mapping  is  used  for  the  synthesis  of  pla¬ 
nar  and  spherical  linkages  in  order  to  state  design  equations 
and  to  provide  a  tool  for  visualizing  the  workspace  and  tra¬ 
jectories  of  the  linkage.  See  Ravani  and  Roth  [14]  and  more 
recent  applications  by  Hayes  [15],  Schrocker  [16]  and  Wu 
and  Ge  [17].  For  spatial  motion,  Study’s  kinematic  mapping 
is  used  to  obtain  simplified  equations  for  analysis  and  synthe¬ 
sis,  see  Husty  et  al.  [18]  and  [19].  However,  the  kinematic 
image  for  the  spatial  motion  is  a  six-dimensional  quadric  and 
that  makes  the  visualization  of  workspaces  and  trajectories 
difficult  for  the  designer. 

In  this  paper,  the  workspace  of  the  linkage  is  visualized 
as  a  set  of  finite  screws  corresponding  to  the  set  of  finite  dis¬ 
placements  of  the  end-effector.  One  interesting  question  is 
whether  the  finite- screw  surfaces  generated  by  a  set  of  task 
positions  can  give  any  information  for  the  synthesis  of  paral¬ 
lel  robots  and,  in  particular,  of  some  overconstrained  closed 
linkages.  Using  Parkin’s  definition  for  pitch  [20],  the  screws 
corresponding  to  finite  displacements  of  some  linkages  can 
form  screw  systems.  Huang  [21]  showed  that  the  single  RR 
chain  forms  a  finite  screw  system  of  third  order;  however, 
the  set  of  finite  displacements  of  the  coupler  of  the  Bennett 
linkage  form  a  cylindroid,  which  is  a  general  2- system  of 
screws  [22].  Baker  [23]  has  also  studied  the  motion  of  the 
peme|t  linkage.  Perez  and  McCarthy  [24]  used  two  arbitrary 
displacements  to  generate  the  cylindroid  of  finite  screws  as¬ 
sociated  to  the  Bennett  linkage  in  order  to  perform  dimen- 


sional  synthesis. 

In  this  paper,  the  focus  is  on  the  simplest  of  the  over¬ 
constrained  linkages,  the  closed  spatial  RPRP  linkage.  This 
linkage  is  one  of  the  Delassus  linkages,  later  studied  by  Wal¬ 
dron  [25]  and  Baker  [26].  Recently,  Huang  [27]  has  shown 
that  the  set  of  screws  corresponding  to  displacements  of  this 
linkage  forms  a  2-screw  system.  We  use  this  result  in  order 
to  synthesize  RPRP  linkages  with  positive  mobility  and  for  a 
given  shape  of  the  screw  system  of  relative  displacements.  In 
order  to  do  so,  we  state  the  design  equations  using  the  Clif¬ 
ford  algebra  of  dual  quaternions  [28].  The  dual  quaternion 
expression  can  be  easily  related  to  the  screw  system  and  it 
is  also  used  to  assign  the  magnitude  to  the  screws  in  order 
to  obtain  the  correspondence  between  the  screw  system  and 
the  trajectory  of  the  end-effector.  The  design  yields  a  single 
RPRP  linkage. 


Fig.  1.  The  RP  serial  chain 


2  Clifford  Algebra  Equations  for  the  Synthesis 

The  approach  used  in  this  paper  for  stating  design  equa¬ 
tions  is  based  on  the  method  of  Lee  and  Mavroidis  [29] .  They 
equate  the  forward  kinematics  of  a  serial  chain  to  a  set  of  goal 
displacements  and  consider  the  Denavit-Hartenberg  parame¬ 
ters  as  variables.  A  more  efficient  formulation  for  our  pur¬ 
poses  consists  of  stating  the  forward  kinematics  of  relative 
displacements  using  the  even  Clifford  subalgebra  of  the  pro¬ 
jective  space  C+(P3),  also  known  as  dual  quaternions  [28]. 

The  Pliicker  coordinates  S  =  (s,c  x  s)  of  a  line  can  be 
identified  with  the  Clifford  algebra  element  S  =  s  +  ec  x  s, 
with  £2  =  0.  Similarly,  a  screw  J  becomes  the  element  J  = 

(1  +/i£)S,  where  ju  is  the  pitch.  Using  the  Clifford  product 
we  can  compute  the  exponential  of  the  screw  |  J, 

0j  0  d  .  0  .  (  .  0  d  0  .  0  .  0 

=  (cos  -  -  -  sm  -  e)  +  (sin  -  +  -  c°s  -£)S  =  cos  -  +  sin  -S. 

(1) 

The  exponential  of  a  screw  defines  a  unit  dual  quater¬ 
nion,  which  can  be  identified  with  a  relative  displacement 
from  an  initial  position  to  a  final  position  in  terms  of  a  rota¬ 
tion  around  and  a  slide  along  an  axis. 


Fig.  2.  The  RPRP  closed  linkage 


2.1  Forward  Kinematics 

For  a  serial  chain  with  n  joints,  in  which  each  joint  can 
rotate  an  angle  0/  and  slide  a  distance  du  around  and  along 
the  axis  S ;,  i  =  1, . . .  ,n,  the  forward  kinematics  of  relative 
displacements  (with  respect  to  a  reference  position)  can  be 
expressed  as  the  composition  of  Clifford  algebra  elements 
corresponding  to  the  motion  of  each  joint.  Let  0o  and  do 
be  the  joint  parameters  of  this  chain  when  in  the  reference 

configuration,  so  we  have  A0  =  (0  —  0o  +  (d  —  do)z).  Then, 
the  movement  from  this  reference  configuration  is  defined  by 


Q(  A0)=e^rs 


^7  c  A8 n  c 

eTb2  ...gT-N 


=  (cos 


A0i 


,  A02 

(cos  — 


■  A0!c  , 

■sin—  S2)- 


A0„ 

.(COS  — 


.  A0! 
sin— Si 

A0„, 


The  RPRP  linkage  has  a  mobility  M  =  —  2  using  the 
Kutzbach-Gruebler  formula;  however,  for  certain  dimen¬ 
sions  of  the  links,  it  moves  with  one  degree  of  freedom. 
Waldron  [25]  shows  that  the  RPRP  linkage  will  have  posi¬ 
tive  mobility  only  if  both  revolute  joints  are  parallel,  and  the 
directions  of  the  prismatic  joints  are  symmetric  with  respect 
to  the  plane  containing  the  revolute  joints. 

The  RPRP  linkage  can  be  seen  as  a  serial  RP  chain  and 
a  serial  PR  chain  joined  at  their  end-effectors.  The  RP  se¬ 
rial  chain  consists  of  a  revolute  joint  followed  by  a  prismatic 
joint.  Figure  1  shows  the  RP  serial  chain  and  Figure  2  shows 
a  sketch  of  the  RPRP  linkage  with  its  axes.  In  the  PR  serial 
chain,  the  order  of  the  joints  in  the  chain  is  switched. 

hi 


sin  Afy*  For  both  the  RP  and  PR  serial  chains,  let  G  =  g  +  eg0  be 

2  the  revolute  joint  axis,  with  rotation  0,  and  H  =  h  +  eh°  the 

(2)  prismatic  joint  axis,  with  slide  d.  Notice  that,  for  synthesis 


purposes,  the  location  of  the  slider,  given  by  h°,  is  irrelevant. 
The  Clifford  algebra  forward  kinematics  equations  for  the 
RP  chain  are 


QRP(AQ,Ad )  =  (cos  —  +  sin—  G)(l+e—  H) 

.  A0  .  A0  ,  .Ad  A0 
=  (cos  —  +  sin  —g)  +  e(  ■ —  cos  Yh+ 

.  A0  0  Ad  .  A0.  , 

smyg ^  +  — sm  — (gxh-g-h)).  (3) 

For  the  PR  chain,  the  only  difference  is  a  negative  sign  in  the 
cross  product.  In  Eq.  (3),  the  angle  and  slide  are  measured 
from  a  reference  configuration. 


2.2  Design  Equations  and  Counting 

Given  a  set  of  m  task  positions  expressed  as  relative  dis¬ 
placements,  P\j  =  cos  +  sin  Piy,  j  =  2, . . .  ,m,  we 
equate  them  to  the  forward  kinematics  equations  in  Eq.  (2), 


Pli 


j  =  2,...,m 


(4) 


in  order  to  create  the  design  equations. 

The  result  is  8  (m  —  1 )  design  equations.  The  design  vari¬ 
ables  that  determine  the  dimensions  of  the  chain  are  the  n 
joint  axes  S?,  i  =  1, . . .  in  the  reference  configuration.  In 
addition,  the  equations  cointain  the  n(m—  1)  pairs  of  joint 
parameters  A0ZJ  =  A0/y  +  A difi,  which  are  also  unknown. 

For  the  RP  (and  similarly  for  the  PR)  serial  chain,  the 
design  equations  are 

QRp(AQj,Adj)=Pij,  j  =  2, . . .  ,m.  (5) 


In  order  to  determine  the  maximum  number  of  task  po¬ 
sitions  for  the  RP  and  PR  chains,  we  apply  Eq.  (6)  to  obtain 
m  =  2.5  task  positions.  Additional  information  is  obtained 
using  Eq.  (7)  to  compute  mr  =  2  task  rotations,  and  mp  =  3 
task  translations.  Hence,  we  can  define  one  arbitrary  rela¬ 
tive  displacement  and  a  second  relative  displacement  whose 
orientation  is  not  general. 


3  Screw  System  for  the  RPRP  Linkage 

In  the  context  of  this  paper,  a  finite- screw  surface  is  a 
ruled  surface  in  which  the  lines  with  their  associated  pitch 
correspond  to  relative  displacements.  A  screw  surface  will 
be  a  screw  system  if  it  is  closed  under  addition  and  scalar 
multiplication,  that  is,  if  every  screw  of  the  set  can  be  written 
as  a  linear  combination  of  screws  belonging  to  it. 


3.1  The  finite  Screw  System  Generated  by  the  Motion 
of  the  RPRP  Linkage 

The  linear  combination  of  two  arbitrary  screws  repre¬ 
senting  relative  displacements  form  a  2- system  known  as  the 
cylindroid,  which  turns  out  to  be  the  manifold  for  the  rel¬ 
ative  displacements  of  the  closed  4R  linkage.  Huang  [27], 
by  intersecting  the  3-systems  associated  with  the  finite  dis¬ 
placements  of  the  RP  and  PR  dyads,  shows  that  the  screw 
surface  of  the  closed  RPRP  linkage  forms  a  2-system  of  a 
special  type,  the  fourth  special  type  according  to  Hunt  [31], 
also  known  as  2-IB  [32].  The  screws  of  this  system  are  par¬ 
allel,  coplanar  screws  whose  pitches  vary  linearly  with  their 
distance. 

The  screw  system  corresponding  to  the  RPRP  linkage 
can  be  obtained  in  a  quite  straightforward  way  by  using  the 
composition  of  relative  displacements  at  each  joint,  as  ex¬ 
pressed  in  Eq.(3).  Let  us  denote  the  resulting  displacements 


The  counting  of  independent  equations  and  unknowns 
defines  the  maximum  number  of  arbitrary  positions  m  that 
can  be  reached,  based  only  on  the  type  and  number  of  joints 
of  the  serial  chain,  see  [30]  for  details.  Consider  a  serial 
chain  with  r  revolute  and  p  prismatic  joints.  The  maximum 
number  of  task  positions  is  given  by 


3  r  +  p  +  6 
6-(r  +  p)' 


(6) 


-  A\i if  A\i/ 

Q*p(A0,  Ad)  =  cos  —  +  sui—Srp  = 

,  A\|/  At  .  A \|/  .  A\\t 

(cos  — - 8—  sin  — — )  +  sin  — —  ( 1 

v  2  2  2  2  v 


A t_ 
2 


tan 


A\ \f 


)S  RP, 


(8) 


where  A\| f  and  At  are  the  angle  about  and  slide  along  the 
screw  axis  Srp  of  the  resulting  relative  displacement.  Ex¬ 
pand  the  product  in  Eq.(3)  and  separate  the  dual  scalar  and 
the  dual  vector  part,  so  that 


For  serial  chains  with  less  than  three  revolute  joints,  the 
structure  of  semi-direct  product  of  the  composition  of  dis¬ 
placements  needs  to  be  considered,  and  the  maximum  num¬ 
ber  of  rotations  mp  needs  to  be  calculated  too.  Assuming  that 
the  orientations  are  given  and  that  both  the  directions  of  the 
re  volute  joints  and  the  angles  to  reach  the  task  orientations 
are  known,  we  can  count,  in  a  similar  fashion,  the  number  of 
translations  mp  that  the  chain  can  be  defined  for. 


At  .  A\i/  A0  Ad  .  A0  _ 

—  8—  sin  — —  =  cos  — —  8— -  sin  — - G  •  H 
2  2  2  2  2 


•  Ay 

sin— ^(1  +8 


A t_ 
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2  '  ~tan^ 


)S  RP  = 


.  A0  Ad  A0  Ad  .  A0^  . . 

sin— -G  +  8— -cos— -H  +  8— -sin— -G  x  H. 
2  2  2  2  2 
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mR 


3  +  r 
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mp 


2r  +  ^  +  3 
3  ~P 


Pa  e  383  of  5§?r  stuc^n&  screw  system,  we  are  only  interested  in 
the  dual- vector  part  of  Eq.(9),  which  defines  the  lines  with 
^  their  associated  pitch. 


The  resulting  screw  is  parameterized  by  joint  variables 
A0  of  the  revolute  joint  and  Ad  corresponding  to  the  pris¬ 
matic  joint.  We  denote  the  expression  in  Eq.(9)  as  the  finite 
kinematic  generator  of  the  screw  system.  In  the  case  of  finite 
displacements,  the  screw  systems  correspond  to  subspaces  of 
a  projective  space,  and  so  we  take  the  unit  screws  as  repre¬ 
sentatives  by  dividing  the  dual  vector  of  Eq.(9)  by  sin^. 
From  the  real  part  of  the  first  equation  in  (9)  we  can  see  that 
sin  ^  =  zb  sin  ^ ,  so  that 


which  is  different  from  zero  when  the  angle  0  and  the  slide  d 
are  independent,  except  for  special  parallel  or  perpendicular 
arrangements  of  the  joints.  Hence,  a  maximal  set  of  indepen¬ 
dent  screws  has  cardinality  3. 

In  order  to  generate  the  screw  system  of  the  RPRP  link¬ 
age,  we  impose  the  motion  constraint  relations  between  the 
joint  variables.  The  conditions  are  derived  in  [27]  from  the 
closure  equations  of  the  linkage,  to  obtain 


—  —Ad 

i(l±e — ^aw^rp  =  ^ £ — ^Ae  H  +  £-— G  x  H.  (10) 
tan  Y  tan  tf  2 

If  we  give  values  to  the  joint  variables  A0  and  Ad,  we 
will  generate  a  screw  surface  of  dimension  3  of  a  special 
type,  in  which  all  the  screws  have  parallel  directions. 

It  is  easy  to  show  that  the  set  of  screws  generated  by  the 
motion  of  the  RP  serial  chain  is  indeed  a  screw  system,  that 
is,  closed  under  addition  and  multiplication  by  scalar.  Gener¬ 
ate  two  unit  screws  S\  and  S2  using  Eq.(10)  and  perform  the 
linear  combination  with  real  constants  k\  and  fe.  We  normal¬ 
ize  the  resulting  screw  by  dividing  by  k\  ±  to  obtain 

(1+es#)S3=*rb;<‘lS|+‘2S2) 

Ml 

=g  +  £(g°  +  £ — ^§-h  +  £— ^g  x  h), 
tan  ~y~  ^ 

(11) 


where 
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Ad2, 
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Adi 

2 
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A  d2 
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-k2 
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A02 


(12) 


hence  the  linear  combination  belongs  to  the  screw  system 
generated  by  the  relative  motion  of  the  RP  chain.  Next  is  to 
calculate  the  dimension  of  this  screw  system.  Notice  that  the 
direction  of  all  unit  screws  generated  is  equal  to  g,  then  we 
just  need  to  look  at  the  dimension  of  the  dual  part.  Generate 
three  screws  with  values  A0?  and  Adi,  f  =  1,2,3.  For  the  three 
vectors  corresponding  to  the  dual  part  to  be  linearly  indepen¬ 
dent,  the  determinant  of  the  column  vector  matrix  must  be 
different  from  zero.  We  obtain  the  value  for  the  determinant 


d  =  J2, 

0  =  ±(02  —  7t), 
tan®  _  sinai2 
d  <212  ±<223 


(14) 


where  the  twist  angle  OC12  and  link  lengths  <212,  <223  are  shown 
in  Figure  2;  the  angles  0,  02  and  slides  d,  J2,  also  shown  in 
Figure  2,  have  to  be  measured  according  to  the  convention 
[27]  from  the  previous  common  normal  line.  The  plus/minus 
sign  corresponds  to  folded  and  unfolded  linkages.  For  our 
purposes,  only  the  third  condition  is  needed.  It  is  important 
to  notice  that  this  condition  applies  to  absolute  values  of  the 
joint  variables,  measured  from  the  previous  common  normal 
line  as  stated  before. 

In  order  to  use  this  relation,  substitute  Ad  =  d  —  do, 
A0  =  0  —  0o  to  apply  Eq.(14)  so  that  the  resulting  expression 
depends  only  on  d,  for  instance.  For  all  possible  values  of  the 
slide  d,  this  generates  a  2-IB  system  of  screws  [31, 32].  This 
can  be  checked  numerically  by  generating  6  random  screws 
and  computing  the  rank  of  the  matrix  that  has  the  screws  as 
columns.  In  this  case,  the  rank  is  2.  It  can  also  be  shown  that 
when  the  third  condition  in  Eq.(14)  holds,  the  determinant  in 
Eq.(13)  is  equal  to  zero. 

The  unit  screws  of  the  system  can  be  related  to  spa¬ 
tial  displacements  if  we  add  the  value  of  the  magnitude  of 
the  screw.  For  finite  displacement  screws,  the  value  of  the 
magnitude  related  to  each  screw  is  unique;  this  is  due  to  the 
fact  that  finite  screw  systems  are  projective  subspaces  [33]. 
The  information  about  the  magnitude  corresponding  to  each 
screw  can  be  extracted  from  the  scalar  part  of  the  dual  quater¬ 
nion  product  in  Eq.(9).  This  equation  relates  the  rotation 
associated  to  the  resulting  screw,  A\\f,  to  the  joint  variables 
of  the  kinematic  chain.  For  the  RPRP  chain,  this  yields 
sin  ^  =  ±  sin  ^  as  noted  before. 

Figure  3  illustrates  the  nonlinear  relation  between  the 
screws  generated  by  an  RPRP  closed  chain  and  the  corre¬ 
sponding  set  of  absolute  positions  of  the  end  effector  (for 
those  positions  we  assume  that  the  reference  configuration  is 
the  identity).  We  use  as  values  for  the  linkage  those  of  an 
example  from  [27]. 
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3.2  The  Finite  Screw  System  Generated  as  a  Linear 
Combination  of  Two  Screws 

It  has  been  shown  that  the  relative  displacements  of  the 
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a  e  38-Po?5?lC^a^n  Senerate  a  2-IB  finite  screw  system.  By  defini- 
896  tion,  this  same  screw  system  can  be  generated  as  the  linear 
(12)  combination  of  two  screws  with  same  direction  and  arbitrary 


Fig.  3.  Screw  system  generated  by  the  RPRP  linkage,  above;  cor¬ 
responding  absolute  displacements,  below 


location  and  (possibly)  finite  pitches.  From  a  synthesis  point 
of  view,  the  key  is  that  this  coincides  with  the  results  of  the 
counting  in  section  2.2.  The  task  positions  defined  for  the 
synthesis  of  the  RP  (or  PR)  chain  are  two  relative  displace¬ 
ments  with  same  direction  and,  in  general,  finite  pitches. 

The  screw  system  is  characterized  by  the  pitch  distribu¬ 
tion  as  a  linear  function  of  the  distance  between  screws  along 
a  common  normal.  In  this  derivation  we  can  parameterize  the 
results  as  a  function  of  the  angle  0  or  the  slide  d. 

Consider  the  screw  Srp  =  (1  +  £p)Srp  =  Srp  +  es^.The 

s  s° 

pitch  is  obtained  by  computing  p  =  s^.s^  in  Eq.(10), 


the  axis,  c rp  =  and  computing  the  norm  of  the  dif¬ 

ference  for  two  of  them, 
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The  slope  of  the  (linear)  pitch  distribution  is  computed 
as 


Pb-Pa 

I|cb-ca||’ 


(19) 


and  for  the  RPRP  chain  we  can  simplify  the  expression  to 


K  =  ± 


COSOC12 


dl  sin2  OC12  +  (a\2  +  d2?>)2 


,  ,  cosai2  .  0o 

do  =  ±  — - sin  — . 

sinai2  2 


(20) 

Notice  that  the  slope  is  constant  and  depends  on  the  initial 
configuration.  The  sign  is  negative  if  dp  <  d\  or  if  tan  2  < 
tan^-. 

The  screw  system  is  related  to  the  displacement  of  the 
RPRP  linkage  by  computing  the  distribution  of  the  magni¬ 
tude  with  respect  to  the  known  pitch,  distance  and  magnitude 
of  the  screws  Sa  and  Sp  used  to  define  the  distribution.  Con¬ 
sider  the  magnitude  of  the  screws  as  m  =  sin  =  db  sin  . 
Knowing  the  magnitude  and  pitch  of  screws  Sa  and  Sp,  we 
can  solve  for  the  magnitude  me  of  a  screw  Sc  using  Eq.(18) 


me  =  ±  sin 


A0C 


2  \/T  +x2’ 

where  the  factor  X  is  simplified  using  Eq.(15)  to  (20)  to 
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For  two  screws  Sa  and  Sp,  the  difference  in  the  pitches  is 
given  by 


HcB-CAH/Mtan^!1  -  ||cc-CA||(pAtan^±pstan^) 

“  ||cb-ca||  7c 

(22) 

The  positive  and  negative  signs  in  the  expression  of  X 
correspond  to  the  sign  of  the  tan  ^  —  tan  ^  factor. 


_kd0  gh  tan^gh  0B  0A 

Pb  Pa—  ^  y^B  dA)  —  ^  Vtan  2  tan  2  3.3  Defining  the  Kinematic  Task  as  a  Finite  Screw  Sys- 

385  of  53^iem 

The  distance  between  two  screws  along  the  common  ft  is  now  possible  to  define  the  screw  system  as  the  input 

normal  is  calculated  by  finding  the  perpendicular  point  on  task  for  the  dimensional  synthesis  of  the  RPRP  closed  chain. 


As  a  linear  subspace,  it  is  easy  to  shape  the  task  at  our  conve¬ 
nience.  Once  we  have  a  satisfactory  screw  system,  any  two 
screws  from  it  can  be  selected  to  perform  the  dimensional 
synthesis  and  to  obtain  a  finite  number  of  solutions  (in  this 
case,  just  one). 

We  have  several  strategies  for  shaping  the  screw  sys¬ 
tem.  For  instance,  we  can  define  a  first  relative  displace¬ 
ment,  5i2  =  cos  ^  +  sin  ^  (S12  +  es^)-  The  rotation  axis  of 
the  displacement,  S12  is  common  to  both  *§12  and  the  second 
relative  displacement.  We  set  S12  =  S13  and  select  a  rotation 
angle  to  define  the  relative  rotation  S13. 

We  can  then  set  the  slope  of  the  pitch  distribution  in  or¬ 
der  to  shape  the  screw  system.  The  pitch  for  the  finite  dis¬ 
placement  screws  is  [20] 


Pii  = 


(23) 


The  equations  corresponding  to  the  dual  part  are  linear  in  the 
moment  of  the  re  volute  joint,  g°, 


g°=- 


sm 


/  Q  A  d\i  ,  A9i;  .  A01; 

7  (q  u  ~  —  (cos  — h  +  sm  —g  x  h)) , 
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1  =  1,2. 
(26) 


Equating  the  solution  of  g°  for  both  relative  displacements, 
we  can  solve  linearly  for  h  as  a  function  of  the  slides  Adi  2, 
Adi  3.  The  relation  between  the  slides  is  given  by  the  pitch 
condition, 
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(27) 


Imposing  ||h||  =  1,  we  can  solve  for  the  slides  to  obtain  one 
solution.  Using  the  same  process,  we  can  solve  for  the  PR 
serial  chain. 


directly  calculated  from  the  dual  quaternion  using  pu  = 

Si  -S^ 

s  .  Similarly,  a  point  on  the  screw  axis  is  calculated  as 


Define  the  slope  of  the  distribution  as  K  =  n^13  £ 12„ ,  accord- 
ing  to  Eq.(20).  If  we  set  the  value  of  K ,  we  can  solve  for 
At  13  in  order  to  define  the  pitch  of  the  second  relative  dis¬ 
placement,  the  location  of  its  screw  axis  being  defined.  This 
is  one  possible  way  of  defining  the  screw  system.  Convert¬ 
ing  from  this  to  absolute  displacements  we  can  easily  check 
whether  the  trajectory  for  the  synthesis  is  acceptable. 

Any  other  strategy  to  basically  define  a  triangle  in  space 
can  be  used.  The  purpose  of  this  is  to  have  a  better  control  on 
the  shape  of  the  trajectory  of  the  linkage  than  the  one  given 
by  just  two  separate  finite  positions. 


5  Examples 

Two  examples  are  presented  below.  The  first  one  is  per¬ 
formed  to  check  the  method,  while  for  the  second  one,  a  fully 
general  task  is  used. 

5.1  First  Example 

In  this  first  case,  the  two  relative  displacements  are  gen¬ 
erated  using  the  RP  chain  presented  in  Huang  [27].  In  this 
example,  the  revolute  joint  is  located  at  the  origin  and  the 
prismatic  joint  is  located  along  the  x  direction  with  a  twist 
angle  of  an  =  ft/5  and  link  length  an  =  5.  We  use  the  loop 
condition  in  Eq.(14)  and  random  values  for  the  slide  to  gen¬ 
erate  the  set  of  relative  displacements  of  the  corresponding 
RPRP  closed  chain.  From  those,  we  randomly  select  two 
displacements,  the  ones  in  Table  1. 

Table  1.  Goal  relative  displacements  for  the  RP  and  PR  chains, 
Huang’s  example 


4  Dimensional  Synthesis  of  the  RPRP  Linkage  for  a 
Prescribed  Screw  System 

The  synthesis  of  the  RP,  and  similarly,  PR  chains,  is  sim¬ 
ple  and  yields  one  solution.  In  general,  the  task  positions 
are  expressed  as  relative  displacements  Qu  =  QiQ[  with 
respect  to  the  first  task  position  Q\ .For  the  RP  or  PR  chains, 
the  maximum  number  of  task  positions  that  we  can  define  is 
i  =  3,  as  explained  in  section  2.2.  Given  an  arbitrary  relative 
displacement  Qn  =  fan  +  Qi2)  +  £fan  +  q?2)  and a  second 
displacement  Q\ 3  =  fa™3  +  ql3)  +  £(q™® +  q®3)  such  that  both 
have  same  direction  and  a  given  pitch  distribution,  we  equate 
them  to  the  forward  kinematics  in  Eq.(3).  We  can  solve  for 
the  direction  of  the  revolute  joint  g  and  the  rotation  angles, 


g  = 


Ql2 

IMI’ 


tan 


A0ii  _  ||Qij I 


Tu 


i  =  2,3. 


Page 

(25) 


(0,0, -0.05,0.99) +8(0.02,0.37, -0.51, -0.03) 
(0,0,  -0.34,0.94)  +8(0.80,2.23,  -3.08,-1.10) 


The  resulting  screw  system  is  shown  in  Figure  4,  where 
the  length  of  each  screw  is  proportional  to  its  pitch.  Also 
the  corresponding  trajectory  of  absolute  displacements,  con- 
isdering  the  reference  position  as  the  identity,  is  shown  in 
same  Figure. 

Using  the  synthesis  procedure  explained  in  Section  4, 
we  obtain  the  RP  and  PR  chains  of  Table  2.  Notice  that  they 
coincide  with  the  values  given  in  [27],  which  means  this  is 
a  folded  RPRP  overconstrained  movable  linkage.  Figure  5 
38$  o?§3S1  ^n^aSe  caching  some  of  the  positions.  Notice 
that  the  shape  of  the  coupler  link  attached  to  the  end  effector 
depends  on  the  position  chosen  as  reference  configuration. 


Fig.  4.  Above,  screw  system  generated  by  S 12  and  5 13  (shown  as 
first  and  last  screws);  below,  corresponding  absolute  displacements 
with  reference  displacement  being  the  identity 


Fig.  5.  RPRP  linkage  reaching  three  positions  of  the  trajectory 


5.2  Second  example 

For  the  second  example,  the  dual  quaternions  in  Table 
3  have  been  generated  as  explained.  S 12  has  been  randomly 
generated,  while  the  rotation  in  £13  is  such  that  it  belongs  to 
the  workspace  of  the  chain. 

We  set  the  location  of  the  second  screw  axis  of  the  rela¬ 
tive  displacement  with  a  point  p13  =  (0.083,2.159,-3.226), 
randomly  generated.  Then  we  choose  a  value  for  the  slope  of 
the  pitch  distribution,  K  =  0.480,  which  allows  us  to  create 
the  second  relative  displacement.  aQe 

The  resulting  screw  system  is  shown  in  Figure  6,  where 


Table  2.  Joint  axes  for  the  RPRP  linkage  at  the  reference  configu¬ 
ration,  Example  1 


Chain 

Revolute  joint  G 

Prismatic 

joint  h 

/  \ 
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013  =  —39.4  J13  =  8.08 


0  +  5.388 

s  \ 

-0.42 

PR  axes 

< 

0+13.08 

>  < 

-0.42 
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-0.81  1 

PR  Joint  vars.  812  =  5.7  d\2  =  1.27 

013  =  39.4  J13  =  8.08 


Table  3.  Goal  relative  displacements  for  the  RP  and  PR  chains 


(0.46,  -0.13,  -0.56,  -0.67)  +8(1.66,0.34,  -0.02, 1.08) 
(0.13,  -0.04,  -0.17,0.98)  +8(0.02,  -0.57,  -0.92,  -0.18) 


the  length  of  each  screw  is  proportional  to  its  pitch.  The  cor¬ 
responding  trajectory  of  absolute  displacements,  conisdering 
the  reference  position  as  the  identity,  is  shown  in  Figure  7. 

We  obtain  one  solution  for  the  RPRP  linkage,  specified 
in  Table  4  as  the  Pliicker  coordinates  of  the  axes  and  the  joint 
variables  to  reach  the  positions. 

Again,  the  dimensions  form  an  overconstrained  mov¬ 
able  RPRP  linkage.  Figure  8  shows  the  chain  reaching  three 
displacements  along  the  trajectory,  considering  the  reference 
displacement  as  the  identity. 

6  Conclusions 

This  papers  presents  the  exact  workspace  synthesis  of  an 
overconstrained  closed  linkage,  the  RPRP.  The  knowledge  of 
the  screw  system  that  corresponds  to  the  finite  displacements 
of  the  linkage  is  used  to  generate  the  workspace  of  the  link¬ 
age,  which  in  turn  ensures  that  the  solutions  of  the  synthesis 
of  the  RP  and  PR  serial  chains  can  be  assembled  to  create  a 
movable  system.  The  counting  of  the  maximum  number  of 
positions  for  the  finite-position  synthesis  of  the  serial  chain 
turns  out  to  fully  define  the  finite  screw  system  of  the  link¬ 
age.  This  provides  an  easy  method  to  shape  the  whole  trajec- 
|or^gf  the  RPRP  linkage  as  an  input  for  the  synthesis  pro- 
’  cess.  Even  though  the  method  targets  the  whole  motion  of 
the  linkage,  the  synthesis  equations  need  to  be  stated  only  at 


Fig.  6.  Screw  system  generated  by  Si 2  and  S13  (first  and  last 
screws) 


the  task  positions  used  to  generate  the  linear  combination  of 
screws,  hence  the  finite-position  synthesis  equations  can  still 
be  used.  The  method  yields  a  single  RPRP  linkage. 
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ABSTRACT 

In  this  paper,  we  explore  the  idea  of  designing  non- 
anthropomorphic,  multi-fingered  robotic  hands  for  tasks  that 
replicate  the  motion  of  the  human  hand.  Taking  as  input  data 
rigid-body  trajectories  for  the  five  fingertips,  we  develop  a 
method  to  perform  dimensional  synthesis  for  a  kinematic  chain 
with  a  tree  structure,  with  three  common  joints  and  five  branches. 

We  state  the  forward  kinematics  equations  of  relative  dis¬ 
placements  for  each  serial  chain  expressed  as  dual  quaternions, 
and  solve  for  all  five  chains  simultaneously  to  reach  a  number  of 
positions  along  the  hand  trajectory  using  a  hybrid  global  numer¬ 
ical  solver  that  integrates  a  genetic  algorithm  and  a  Levenberg- 
Marquardt  minimization. 

Although  the  number  of  candidate  solutions  in  this  problem 
is  very  high,  the  use  of  the  genetic  algorithm  lets  us  to  perform  an 
exhaustive  exploration  of  the  solution  space  and  retain  a  subset 
of  them.  We  then  can  choose  some  of  the  solutions  based  on  the 
specific  task  to  perform.  Note  that  these  designs  could  exactly 
match  the  task  while  having  a  finger  design  radically  different 
from  that  of  the  human  hand. 


*  Address  all  correspondence  to  this  author. 


NOMENCLATURE 

r  Number  of  revolute  j  oints . 
b  Number  of  kinematic  chains. 
n  Number  of  j  oints . 
m  Number  of  task  positions. 

INTRODUCTION 

There  are  many  applications  for  which  a  robotic  system  is 
needed  to  work  in  human  environments  and  to  perform  tasks  that 
are  designed  for  the  human  hand.  In  most  cases,  the  solution 
adopted  for  grasping  and  manipulation  consists  of  anthropomor¬ 
phic  robotic  hands,  which  imitate  to  certain  extent  the  topology 
and  joint  location  of  the  human  hand,  see  [1]  for  a  review  of  ap¬ 
plications  and  concept  definition. 

It  is  difficult  to  match  the  complexity  of  the  human  hand, 
commonly  accepted  to  have  26  degrees  of  freedom  when  count¬ 
ing  the  motion  at  the  wrist  and  the  pronation/supination  of  the 
forearm.  The  anthropomorphic  design  must  include  a  complex 
mechanical  system,  actuation  and  sensing  in  a  small  space  [2]. 
In  order  to  reduce  complexity,  actual  designs  limit  the  active  de¬ 
grees  of  freedom  in  many  cases,  through  simplification  of  the 
mechanical  structure  or  by  designing  underactuated  hands  [3]. 

The  design  of  simplified  grippers  limits  the  tasks  that  the 
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robotic  end-effector  can  perform  to  some  grasping  and  manipu¬ 
lation  actions.  In  order  to  perform  some  of  the  more  complex 
functions  of  the  human  hand  (not  only  grasping  and  manipula¬ 
tion,  but  also  perception  through  surface  exploration),  it  seems 
that  an  end-effector  with  several  fingers  may  be  needed.  How¬ 
ever,  it  may  not  be  necessary  that  the  robotic  fingers  mimic  those 
of  the  human  hand. 

Kinematic  design  of  robotic  hands  has  focused  on  the  de¬ 
sign  of  individual  motion  of  fingers  or  parts  of  the  hand.  Dai  and 
Wang  [4]  use  kinematic  synthesis  to  design  a  spherical  mech¬ 
anism  to  act  as  a  the  palm  of  a  metamorphic  hand.  Van  Var- 
seveld  and  Bone  [5]  designed  a  finger  mechanism  for  a  non- 
anthropomorphic  dexterous  hand.  Walker  et  al.  [6]  also  design 
planar  linkages  for  the  fingers  of  a  non- anthropomorphic,  dexter¬ 
ous  hand.  Schafer  and  Dillman  [7]  present  the  kinematic  design 
of  a  humanoid  robotic  wrist. 

In  this  paper,  we  explore  the  idea  of  designing  the  full 
non- anthropomorphic,  multi-fingered  robotic  hand  for  tasks  that 
replicate  the  motion  of  the  human  hand  at  the  fingertips.  Taking 
as  input  data  rigid-body  trajectories  for  the  human  fingertips,  we 
develop  a  method  to  perform  dimensional  synthesis  for  a  kine¬ 
matic  chain  with  a  tree  structure,  with  three  common  joints  and 
five  branches.  As  a  whole,  the  process  entails  the  simultaneous 
solution  of  five  serial  chains,  two  of  them  having  four  indepen¬ 
dent  joints  and  three  of  them  with  five  independent  joints,  plus 
three  common  joints  for  all  of  them.  The  total  degrees  of  free¬ 
dom  of  the  non-anthropomorphic  hand  design  is  26,  similar  to 
the  human  hand. 

We  state  the  forward  kinematics  equation  of  relative  dis¬ 
placements  for  each  serial  chain  expressed  as  dual  quaternions, 
and  solve  for  all  five  chains  simultaneously  to  reach  a  number 
of  positions  along  the  hand  trajectory.  The  synthesis  of  spatial 
serial  chains  for  up  to  five  degrees  of  freedom  was  developed 
in  [8].  We  use  a  similar  methodology  together  with  a  hybrid 
global  numeric  solver,  composed  of  a  genetic  algorithm  paired 
with  a  Levenberg-Marquardt  local  solver.  For  the  tree  kinematic 
structure,  a  high  number  of  positions  can  be  defined  to  perform 
exact  synthesis,  obtaining  a  good  approximation  for  the  desired 
trajectory. 

Finding  the  complete  solution  set  for  the  synthesis  of  com¬ 
plex  kinematic  chains  is  an  unsolved  problem.  Only  dyads,  such 
as  the  RR  kinematic  chain  [9],  [10],  and  some  triads  with  par¬ 
ticular  characteristics  [11],  have  been  fully  studied.  Even  the  3R 
kinematic  chain  has  not  been  completely  solved  with  a  closed 
algebraic  expression  [12].  The  complexity  of  a  tree-like  kine¬ 
matic  chain  with  a  total  of  26  revolute  joints,  and  our  numerical 
results,  lead  to  believe  that  there  will  be  a  very  large  amount  of 
solutions.  This  is  an  issue  that  has  been  found  before  even  for 
simpler  kinematic  chains,  see  [11].  In  order  to  deal  with  this, 
additional  constraints  may  be  added  to  help  in  the  selection  of 
the  final  design,  depending  on  the  task.  We  present  one  of  the 
solutions  for  a  hand  task  that  was  synthetically  generated.  The 


designs  could  exactly  match  the  task  while  having  a  finger  design 
radically  different  from  that  of  the  human  hand. 


KINEMATIC  SYNTHESIS 

The  goal  of  the  dimensional  kinematic  synthesis  is  to  find 
the  location  and  orientation  of  a  set  of  joint  axes  able  to  perform 
a  given  motion,  where  the  number  and  type  of  joints  are  pre¬ 
defined.  In  this  paper,  we  follow  the  original  idea  of  [13]  of 
using  the  forward  kinematics  equations  of  the  kinematic  chain, 
but  formulated  as  relative  displacements  and  expressed  as  dual 
quaternions,  see  [8]  for  a  complete  description  of  this  approach. 

The  input  data  for  the  synthesis  are  the  m  —  1  relative  trans¬ 
formations  P\j  =  cos  +  sin  P\j,  j  =  2, . . .  ,m,  defining 
the  task;  the  output  are  the  Pliicker  coordinates  S;  =  s;  +  £s?  = 
Si  +  £C/  x  s/,  i  =  1 , . . . ,  n,  of  the  n  joints  that  define  the  kinematic 

chain,  and  also  the  joint  variables  A Oij  =  6jj  —  OjQ  +  {djj  —  djo)£, 
i  =  1 j  =  2, . . .  ,m,  used  to  reach  the  task  positions,  mea¬ 
sured  from  the  reference  configuration. 

Forward  Kinematics 

Given  a  kinematic  serial  chain  with  n  joints,  we  can  write 
the  kinematics  equations  using  the  product  of  exponentials  of  the 
screws  corresponding  to  the  joint  axes,  as  described  in  [14].  In 
this  paper,  instead  of  calculating  the  exponentials  using  matrix 
algebra,  we  do  the  exponentials  for  the  Clifford  even  subalgebra 
of  the  projective  space,  in  which  the  unit  elements,  also  known  as 
dual  quaternions,  express  spatial  displacements.  The  exponential 
of  a  screw  represented  by  the  Clifford  algebra  element  J  =  ( 1  + 
/i£)S,  where  /I  is  the  pitch  relating  the  slide  d  and  the  rotation  6 
along  and  about  the  screw,  and  S  is  the  screw  axis,  yields  a  finite 
displacement, 


9  j  f  6  d  .  0  0 

£2  =(cos  2  ~  2Sm  2^  +  (Sm2 

0  .  0r 

=  cos  —  +sm  —5. 


d 

2 


cos  — £)S 


(1) 


For  a  serial  chain  with  ft  joints,  in  which  each  joint  can  rotate 
an  angle  0/  and  slide  a  distance  around  and  along  the  axis  S *, 
i  =  1 , . . . ,  n,  we  calculate  the  forward  kinematics  of  relative  dis¬ 
placements  (with  respect  to  an  arbitrary  reference  configuration), 


Q(A0)=e 


c 


(2) 


.  Aft  .Aft  A6„  .  A6>„ 

=  (cos  —  +  sin  —Si )  •  •  •  (cos  —  +  sin  — S  „), 


(3) 


2 
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where  Afy  =  (fy —  Go  +  (©  —  do) e)  contains  the  joint  variables, 
as  relative  values  with  respect  to  the  joint  parameters  of  the  chain 
$o  and  d0  when  in  the  reference  configuration. 


Synthesis  Design  Equations 

The  dimensioning  of  the  articulated  system  has  to  be  done 
so  that  the  forward  kinematics  equations  in  Eqn.(3)  can  reach  all 
the  desired  task  positions  P\j, 


P\j  =  e 


0_lj_  A02/ 


S2. 


(4) 


This  results  in  8(m  —  1)  design  equations.  The  design  vari¬ 
ables  that  determine  the  dimensions  of  the  chain  are  the  n  joint 
axes  Sf,  i  =  in  the  reference  configuration.  In  addi¬ 

tion,  the  equations  contain  the  n(m  —  1)  pairs  of  joint  parameters 
A Ojj  =  A6jj  +  AdjjS,  which  are  also  unknown. 

One  important  question  when  stating  design  equations  such 
as  those  in  Eqn.(4)  is  how  many  task  positions  can  be  imposed  for 
the  kinematic  chain  to  reach  exactly.  This  is  easily  computed  by 
comparing  the  number  of  independent  equations  and  unknowns, 
see  [15]  for  details.  The  maximum  number  of  task  positions  m  to 
reach  exactly  depends  on  the  topology  of  the  chain, 


3r  +  /?  +  6  —  c 

m  =  — 7 — 7 - t— 

6  —  (r  +  p) 


(5) 


where  r  is  the  number  of  revolute  joints,  p  is  the  number  of  pris¬ 
matic  joints  and  c  is  any  additional  constraint  in  the  general  po¬ 
sition  of  the  axes  or  in  the  joint  variables. 

Setting  the  design  equations  for  the  maximum  number  of 
task  positions  ensures  that  the  number  of  solutions  is  finite.  How¬ 
ever,  the  resultant  system  of  equations  has  a  total  degree  that 
grows  quickly  with  the  number  of  joints,  and  so  the  number  of 
solutions  may  be  very  high. 


NON-ANTHROPOMORPHIC  HAND  SYNTHESIS 

In  this  paper,  we  are  interested  in  the  synthesis  of  robotic 
grippers  for  human-hand  tasks.  We  don’t  impose  any  limitation 
on  the  link  dimensions  or  placement.  The  resulting  design  should 
be  able  to  perform  a  given  human-hand  task  while  having  a  non- 
anthropomorphic  aspect. 


Hand  Task  Generation 

In  this  paper,  we  want  to  design  a  mechanical  system  to 
replicate  some  of  the  motions  of  the  human  hand.  In  a  first  stage, 
we  define  this  task  using  a  robotics  description  of  the  human 
skeleton,  considering  dimensions  from  the  literature  [16,  17]. 
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FIGURE  1 :  HAND  SKELETON  USED  TO  GENERATE  POSI¬ 
TIONS. 


TABLE  1 :  JOINT  ANGLES  USED  TO  GENERATE  THE  TRA¬ 
JECTORY. 


Chain 

Hand  task  revolute  joint  limits  [0min-6max] 

Common 

[-90°  ,90°  ] ,  [-90°  ,90°  ] ,  [-90°  ,90°  ] 

Index 

[-10°,90o],[-10o,10o],[0o,100o],[0o,90°] 

Middle 

[-10°,90o],[-10o,10o],[0o,100o],[0o,90°] 

Third 

[-10o,10o],[0o,20o],[-10o,10o],[0o,100o],[0°,90o] 

Fourth 

[-10o,10o],[0o,20o],[-10o,10o],[0o,100o],[0°,90o] 

Thumb 

[-25°,25o],[-25o,25o],[-10o,10o],[0o,70°], 

[-10°,85°] 

The  task  is  defined  as  a  series  of  finite  positions  and  orientations 
for  each  fingertip,  creating  by  assigning  a  set  of  joint  variables  to 
the  kinematic  model.  Figure  1  shows  the  draft  of  the  hand  skele¬ 
ton  used  to  generate  the  positions.  The  trajectories  are  generated 
from  random  positions  with  the  joint  angle  ranges  defined  in  Tab. 
1  to  simplify  convergence. 

System  Topology 

As  input  topology,  we  define  a  tree-like  kinematic  chain  with 
three  common  joints  in  series  at  the  base,  connected  to  five  serial 
chains  arranged  in  parallel,  two  of  them  with  four  degrees  of 
freedom  and  three  of  them  with  five  degrees  of  freedom.  This 
structure  follows  the  generally  accepted  joint  arrangement  of  the 
human  hand  plus  the  wrist  motion.  Figure  2  shows  the  topology 
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FIGURE  2:  TOPOLOGY  OF  THE  KINEMATIC  CHAIN. 


of  the  kinematic  chain  to  be  synthesized. 


Design  Equations 

We  state  the  design  equations  adapting  Eq.(4)  to  our  par¬ 
ticular  topology.  For  each  one  of  the  serial  chains  in  the  parallel 
arrangement  we  state  a  set  of  design  equations,  to  obtain  the  total 
system  of  equations 


*  bl< 


F?j=e* 


[eMr  2eW~  3  eW* 


Pkxj=e 


°lj  C  AG2j  r  ^37  C 


A03/ 


<isk 

1  9  3/l 


AdJj. 

e^si,k&{  0,1}, 

Abb  , 

e  2  s*,k  G  {2,3,4}, 

7  =  2, . . .  ,m.  (6) 


Here  k  identifies  the  kinematic  chain  and  7  =  2, . . . ,  m  is  the  index 
of  the  task  position. 

Each  of  the  five  serial  chains,  taken  individually,  has  a  total 
of  seven  or  eight  degrees  of  freedom.  Even  though  the  exact  di¬ 
mensional  synthesis  does  not  apply  to  serial  chains  with  six  or 
more  degrees  of  freedom,  the  fact  of  having  three  common  de¬ 
grees  of  freedom  in  this  case  allows  us  to  state  the  design  equa¬ 
tions  of  the  system  as  a  whole.  Using  Eq.(5)  with  r  =  26  revolute 
joints,  three  of  them  being  common  to  all  the  branches,  we  ob¬ 
tain  that  we  can  solve  exactly  for  a  task  defined  by  m  =  27  finite 
positions  for  each  finger.  This  gives  a  total  of  156  structural  pa¬ 
rameters  and  676  joint  variables  in  our  design  equations. 

Notice  that,  even  though  each  individual  finger  have  more 
than  six  degrees  of  freedom,  the  tree-like  architecture  is  not 
kinematically  redundant  when  defining  the  motion  of  several 
branches  at  the  same  time. 


KINEMATIC  SOLVER 

In  order  to  deal  with  the  big  system  of  equations  presented 
in  the  previous  section,  the  selected  solver  tries  to  minimize  the 
error  in  Eqn.(6)  for  each  kinematic  chain.  In  principle,  problems 


with  selection  of  the  metric  does  not  apply  to  this  case,  as  we  are 
targeting  exact  synthesis. 

Solver  Implementation 

Genetic  algorithms  have  already  been  used  in  many  kine¬ 
matic  problems  [18, 19],  however  the  complexity  of  the  problem 
that  we  are  tying  to  solve  is  especially  challenging.  Through  the 
use  of  Clifford  algebra  the  number  of  equations  and  variables  can 
be  reduced  in  comparison  with  the  matrix  algebra  approach.  This 
allows  the  entire  equation  system  to  be  represented  with  only  832 
input  variables  between  both  structural  and  joint  variables. 

The  genetic  algorithm  performs  an  exhaustive  exploration  of 
the  solution  space  in  order  find  a  solution.  However,  due  to  com¬ 
plexity  of  the  system  a  pure  genetic  algorithm  has  trouble  with 
convergence  past  a  certain  fitness.  This  problem  was  overcome 
but  changing  the  solution  space  used  by  genetic  algorithm  to  the 
local  minima  of  the  entire  solution  space.  The  local  minima  are 
found  by  a  Levenberg-Marquardt  minimizer. 

The  genetic  algorithm  chromosomes  consist  of  sets  of  vari¬ 
ables  that  belong  to  the  full  solution  space.  However  after  gener¬ 
ation  they  are  then  converged  on  a  local  minima.  This  local  con¬ 
vergence  is  also  done  when  chromosomes  are  crossed  between 
each  other  or  mutated  to  ensure  each  chromosome  always  repre¬ 
sents  a  local  minima  of  the  search  space. 

The  fitness  is  calculated  as  the  inverse  of  the  sum  of  the  er¬ 
ror  from  Eqn.(6).  This  makes  the  fitness  a  continuous  positive 
function,  which  allows  roulette-wheel  selection  when  choosing 
pairs  from  the  genetic  algorithm  population  to  crossover.  It  also 
converts  the  genetic  algorithm  to  a  maximization  problem.  The 
crossover  rate  is  kept  low  to  encourage  diversity  in  the  popula¬ 
tion,  since  the  strong  convergence  is  provided  by  the  Levenberg- 
Marquardt  minimizer. 

To  avoid  explicit  constraints  the  chromosomes  are  generated 
in  the  proximity  of  the  ideal  kinematic  solution.  This  acts  as  a 
soft  limit  for  the  possible  shapes  of  the  robotic  hand.  It  is  not  un¬ 
usual  for  the  Levenberg-Marquardt  minimizer  to  move  far  from 
the  generation  space  and  find  extremely  non- anthropomorphic 
solutions. 

The  solver  is  executed  until  the  fitness  surpasses  the  value  of 
1010  at  which  it  is  considered  to  have  arrived  to  a  solution.  The 
error  is  attributed  to  the  imprecision  in  the  computer  representa¬ 
tion  of  real  numbers. 

Dimension  of  the  Equation  Set 

The  equations  and  variables  obtained  from  Eqn.(6)  are  not 
all  linearly  independent;  each  dual  quaternion  has  eight  compo¬ 
nents,  but  in  order  to  be  unit  quaternions,  two  constraints  are 
imposed  in  each  of  them.  Similarly,  only  four  of  the  six  Pliicker 
components  of  each  line  are  independent.  Equation  (7)  gives  the 
number  of  variables  and  independent  variables  represented  as  v 
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and  xo  respectively,  while  Eqn.(8)  gives  the  number  of  equations 
and  independent  equations  represented  as  /  and  /q. 

v  =r(  6  +  (m—  1)) 

x°  =r(A+{m  —  1))  (7) 


f=8b(m—  1) 

/°  =6b(m  —  1)  (8) 

Notice  that  the  number  of  positions,  m,  needed  can  be  also 
obtained  by  imposing  f°  =  v°  and  solving. 

Kinematic  Configurations 

It  is  interesting  to  note  is  that  it  is  possible  to  solve  a  reduced 
version  of  the  kinematic  chain  tree.  As  long  as  the  expression  at 

Eq.(9), 

6b-r>0  (9) 

which  is  obtained  from  f°  =  x°  and  imposing  m  >  0,  is  met,  the 
kinematic  configuration  will  be  solvable.  Table  2  shows  all  the 
possible  variations  of  the  tree-like  kinematic  chain  that  can  be 
solved. 

The  most  interesting  configurations  to  solve  is  the  5  kine¬ 
matic  chains  at  once  to  minimize  the  needed  positions  and  the 
configuration  with  only  two  4R  branches  to  reduce  the  amount 
of  variables  needed.  Afterwards  the  remaining  fingers  can  be 
solved  individually  because  the  common  revolute  joints  move¬ 
ment  is  known.  There  are  still  other  systems  that  can  be  solved 
that  are  a  compromise  between  number  of  positions  and  the  num¬ 
ber  of  variables  needed. 

RESULTS 

Table  3  contains  runtime  information  of  example  solutions. 
These  results  were  obtained  on  an  Intel® Core™i7- 870  CPU  at 
2.93GHz.  The  algorithm  has  a  parallel  nature  and  can  be  adjusted 
to  run  on  supercomputers  or  other  distributed  computing  systems 
to  increase  the  calculation  speed.  An  example  of  a  solution  is 
shown  in  Fig. 3.  An  example  of  the  solver’s  converge  can  be  seen 
in  Fig. 4. 

Best  results  have  been  obtained  with  a  population  size  of  100 
and  using  very  different  positions  in  the  task.  The  computational 
time  and  generations  needed  to  find  a  solution  vary  greatly  with 
different  sets  of  positions.  To  minimize  the  computational  time 
it  is  important  to  use  very  different  positions  as  it  simplifies  the 
convergence  of  the  algorithm. 


TABLE  2:  SOLVABLE  SYSTEMS  OF  EQUATIONS  FOR  DIF¬ 
FERENT  COMBINATIONS  OF  CHAINS. 


b 

r 

V 

/ 

m 

Notes 

5 

26 

832 

1092 

27 

Full  model 

4 

22 

1100 

1452 

45 

(5R,  5R,  5R,  4R)  fingers 

4 

21 

714 

938 

29 

(5R,  5R,  4R,  4R)  fingers 

3 

17 

1258 

1666 

69 

(5R,  5R,  4R)  fingers 

3 

16 

608 

800 

33 

(5R,  4R,  4R)  fingers 

2 

11 

550 

726 

45 

(4R,  4R)  fingers 

1 

5 

130 

170 

21 

5R  finger,  common  solved 

1 

4 

56 

72 

9 

4R  finger,  common  solved 

TABLE  3:  SOLVER  EXECUTION  INFORMATION. 


Solution 

Generations 

Time  (hours) 

1 

13 

20.8 

2 

5 

20.5 

3 

10 

16.5 

4 

12 

85.0 

CONCLUSIONS 

In  this  paper  we  present  a  method  for  the  kinematic  syn¬ 
thesis  of  tree-like  articulated  systems,  with  an  application  in  the 
design  of  a  robot  to  perform  human-hand  tasks.  From  the  dimen¬ 
sional  synthesis  point  of  view,  the  interesting  results  is  that,  due 
to  the  tree  structure,  we  can  synthesize  general  serial  chains  with 
more  than  five  degrees  of  freedom.We  may  conclude  from  this 
that,  despite  common  believe  that  the  human  hand  is  a  redundant 
mechanical  system,  it  may  not  be  so  when  we  consider  a  task  in 
which  all  fingers  must  act. 

Dimensional  synthesis  for  articulated  systems  like  the  one 
presented  here,  with  a  high  number  of  joints,  yield  many  solu¬ 
tions.  A  good  process  is  required  in  order  to  select  from  the 
pool  of  candidates.  For  instance,  for  the  application  presented 
in  this  paper,  additional  constraints  could  be  imposed  either  in 
the  solving  process  or  in  the  postprocessing  phase  in  order  to 
find  a  suitable  design.  Among  others,  we  can  cite  size  or  loca¬ 
tion  restrictions  for  exoskeletons  mounted  on  the  human  hand,  or 
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(a)  Full  Hand 


(b)  Index  Finger 


FIGURE  4:  CONVERGENCE  OF  A  SOLVER  EXECUTION. 
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(c)  Middle  Finger  (d)  Third  Finger 


(e)  Fourth  Finger  (f)  Thumb 

FIGURE  3:  OVERVIEW  OF  A  SOLUTION  FOUND. 


dexterity  conditions  at  given  configurations  for  manipulation  in 
human  environments.  Future  work  will  also  include  task  veloci¬ 
ties  and  accelerations  in  order  to  define  grasping  actions. 


REFERENCES 

[1]  Siciliano,  B.,  and  Khatib,  O.,  2008.  Handbook  of  Robotics. 
Springer,  Berlin. 

[2]  Soto  Martell,  J.,  and  Gini,  G.,  2007.  “Robotic  hands:  De¬ 
sign  review  and  proposal  of  new  design  process”.  World 
Academy  of  Science,  Engineering  and  Technology,  26. 

[3]  Briglen,  L.,  Laliberte,  T.,  and  Gosselin,  C.,  2008.  Under¬ 
actuated  Robotic  Hands.  Springer. 

[4]  Dai,  J.,  and  Wang,  D.,  2007.  “Geometric  analysis  and  syn¬ 
thesis  of  the  metamorphic  robotic  hand”.  ASME  Journal  of 
Mechanical  Design,  129 ,  pp.  1 191-1 197. 

[5]  van  Varseveld,  R.,  and  Bone,  G.,  1999.  “Design  and 
implementation  of  a  lightweight,  large  workspace  non- 
anthropomorphic  dexterous  hand”.  ASME  Journal  of  Me¬ 
chanical  Design,  121. 

[6]  Ramos,  A.,  Gravagne,  I.,  and  Walker,  I.,  2006.  “Goldfin- 
ger:  A  non-anthropomorphic,  dextrous  robot  hand”.  ASME 
Journal  of  Mechanical  Design,  128 ,  pp.  815-819. 

[7]  Schaffer,  C.,  and  Dillmann,  R.,  2001.  “Kinematic  design 
of  a  humanoid  robot  wrist”.  Journal  of  Robotic  Systems, 
18(12),  pp.  747-754. 

[8]  Perez  Gracia,  A.,  and  McCarthy,  J.  M.,  2006.  “The  kine¬ 
matic  synthesis  of  spatial  serial  chains  using  Clifford  alge¬ 
bra  exponentials”.  Proceedings  of  the  Institution  of  Me¬ 
chanical  Engineers,  Part  C,  Journal  of  Mechanical  Engi¬ 
neering  Science,  220(7),  pp.  953-968. 

[9]  Perez,  A.,  and  McCarthy,  J.  M.,  2003.  “Dimensional  syn¬ 
thesis  of  bennett  linkages”.  ASME  Journal  of  Mechanical 
Design,  125(1),  pp.  98-104. 

[10]  Brunnthaler,  K.,  2006.  “Synthesis  of  4r  linkages  using  kine¬ 
matic  mapping”.  PhD  thesis,  Institute  for  Basic  Sciences 


6  Copyright  ©  201 1  by  ASME 


Page  396  of  531 


and  Engineering,  University  of  Innsbruck,  Innsbruck,  Aus¬ 
tria. 

[11]  Su,  H.,  McCarthy,  J.  M.,  and  Watson,  L.  T.,  2004.  “Gener¬ 
alized  linear  product  homotopy  algorithms  and  the  compu¬ 
tation  of  reachable  surfaces”.  ASME  Journal  of  Computers 
and  Information  Science  and  Engineering,  4(3),  pp.  226- 
235. 

[12]  Lee,  E.,  Mavroidis,  C.,  and  Merlet,  J.  P.,  2004.  “Five 
precision  point  synthesis  of  spatial  rrr  manipulators  using 
interval  analysis”.  ASME  Journal  of  Mechanical  Design, 
126(5),  pp.  842-850. 

[13]  Lee,  E.,  and  Mavroidis,  C.,  2002.  “Solving  the  geomet¬ 
ric  design  problem  of  spatial  3r  robot  manipulators  using 
polynomial  homotopy  continuation”.  ASME  Journal  of  Me¬ 
chanical  Design,  124(4),  pp.  652-661. 

[14]  Murray,  R.  M.,  Li,  Z.,  and  Sastry,  S.  S.,  1994.  A  Mathe¬ 
matical  Introduction  to  Robotic  Manipulation.  CRC  Press, 
Inc.,  Boca  Raton,  FL. 

[15]  Perez,  A.,  and  McCarthy,  J.  M.,  2004.  “Dual  quaternion 
synthesis  of  constrained  robotic  systems”.  ASME  Journal 
of  Mechanical  Design,  126(3),  pp.  425-435. 

[16]  Fadi  J.  Beijjani,  J.  M.  F.  L.,  1989.  Basic  Biomechanics  of 
the  Musculoskeletal  System.  Lea  &  Febiger,  ch.  Biome¬ 
chanics  of  the  hand,  pp.  275-304. 

[17]  Chang,  L.,  and  Matsuoka,  Y.,  2006.  “A  kinematic  thumb 
model  for  the  act  hand”.  In  Robotics  and  Automation,  2006. 
ICRA  2006.  Proceedings  2006  IEEE  International  Confer¬ 
ence  on,  pp.  1000  -1005. 

[18]  Chocron,  O.,  and  Bidaud,  R,  1997.  “Evolutionary  algo¬ 
rithms  in  kinematic  design  of  robotic  systems”.  In  Intel¬ 
ligent  Robots  and  Systems,  1997.  IROS  ’97.,  Proceedings 
of  the  1997  IEEE/RSJ  International  Conference  on,  Vol.  2, 
pp.  1111 -1117  vol.2. 

[19]  Yang,  Y.,  Peng,  G.,  Wang,  Y.,  and  Zhang,  H.,  2007.  “A  new 
solution  for  inverse  kinematics  of  7-dof  manipulator  based 
on  genetic  algorithm”.  In  Automation  and  Logistics,  2007 
IEEE  International  Conference  on,  pp.  1947  -1951. 


7 


Copyright  ©  2011  by  ASME 


Page  397  of  531 


Design  of  a  Robotic  Hand  and  Simple  EMG  Input  Controller  with  a 
Biologically-Inspired  Parallel  Actuation  System  for  Prosthetic 

Applications 

Dr.  Anthony  L.  Crawford,  Member ;  IEEE ,  Jeffrey  Molitor,  Member ;  IEEE, 

Dr.  Alba  Perez-Gracia  Member,  IEEE,  Dr.  Steve  C.  Chiu,  Member,  IEEE 


Abstract — This  paper  presents  the  mechatronic  design  of  a 
robotic  hand  for  prosthetic  applications.  The  main 
characteristic  of  this  robotic  hand  is  its  biologically-inspired 
parallel  actuation  system,  which  is  based  on  the 
behavior/strength  space  of  the  Flexor  Digitorum  Profundus 
(FDP)  and  the  Flexor  Digitorum  Superficialis  (FDS)  muscles. 
The  design  separates  the  strength  space  of  the  FDS  and  FDP 
muscles  into  a  lighter  strength  region  where  finer  manipulation 
and  general  approach  tasks  are  executed,  and  a  higher  strength 
region  where  the  more  robust  grasps  are  achieved.  Two 
parallel  actuator  types  and  kinematic  structures  are  designed 
to  complement  the  requirements  of  both  strength  space  regions. 

This  unique  structure  is  intended  to  be  driven  by 
electromyographical  (EMG)  signals  captured  at  the  surface  of 
the  skin.  The  direct  relation  between  signal  and  actuation 
system  lends  itself  well  to  interpreting  the  EMG  signals  from 
the  FDP  and  FDS  muscles  into  effective  task  execution,  with  the 
goal  of  helping  the  user  to  achieve  a  good  approximation  of  the 
full  capabilities  associated  with  the  human  hand,  without 
compromising  strength,  dexterity,  appearance,  or  weight; 
which  are  common  issues  associated  with  prosthetic  hands. 

The  designed  finger’s  capability  of  having  a  strength  space 
similar  to  that  of  the  FDS  and  FDP  muscles  is  validated  via 
direct  inputs  from  a  power  supply  and  then  via  a  controller 
using  an  actual  EMG  signal  input  from  the  human  forearm. 
The  controller  is  a  simple  feed  forward  system  at  this  point  in 
the  research  but  provides  the  appropriate  framework  to 
integrate  more  elaborate  control  schemes  and  EMG  signal 
conditioning  as  this  portion  of  the  research  area  matures. 

Keywords  -  Prosthetics,  Parallel  Actuation  Structure,  Robotics, 
Hand,  FDS  and  FDP  Muscles 

I.  Introduction 

THERE  Have  been  many  different  approaches  taken  in 
the  development  of  an  effective  prosthetic  hand.  These 
varying  strategies  often  find  themselves  focusing  on  one  of 
the  following  categories:  implementing  a  new  actuator  type 
[11-16],  developing  a  more  effective  kinematic  structure 
[18,23,34],  integrating  effective  compliance  [18,23-25], 

Manuscript  received  May  24,  2010.  this  research  was  performed  under 
an  award/contract  from  Telemedicine  Advanced  Technology  Research 
Center  (TATRC),  of  the  U.S.  Army  Medical  Research  and  Materiel 
Command  (USAMRMC)  of  the  U.S.  Department  of  Defense. 

A.  L.  Crawford  was  a  Ph.D.  student  at  Idaho  State  University,  Pocatello, 
ID  83209  (e-mail:  crawanth@isu.edu) 

J.  Molitor  is  a  Masters  student  at  Idaho  State  University,  Pocatello,  ID 
83209  (e-mail:  molij eff@isu.edu) 

M.  A.  Perez-Gracia  is  an  assistant  professor  at  Idaho  State  University, 
Pocatello,  ID  83209  and  is  currently  at  Institut  de  Robotica  i  Informatica 
Industrial  (UPC/CSIC),  Barcelona,  Spain  (e-mail:  perealba@isu.edu) 


generating  effective  control  strategies  [25-31],  and 
interpreting/conditioning  input  signals  [25].  Advances  in 
these  areas  have  resulted  in  robotic  hands  that  perform  many 
tasks  with  a  high  similarity  to  that  of  the  human  hand,  such 
as  the  DLR  hand  [33],  I-Limb  hand  [11],  Shadow  hand  [12], 
and  Fluidhand  [32]  to  name  a  few.  However,  a  prosthetic 
hand  that  is  nimble,  quick,  strong,  lightweight,  quiet,  and 
efficient  [1]  has  yet  to  be  achieved. 

The  primary  reason  for  the  current  state  of  prosthetic 
hands  has  been  the  complexity  associated  with  the  human 
hand  as  a  result  of  its  multiple  bones  and  joints  (Fig  1).  This 
is  further  compounded  by  the  fact  that  the  human  hand  as  a 
functioning  unit  does  not  just  embody  the  palm  and  its  digits 
but  also  the  wrist,  forearm  muscles,  nervous  system,  and  the 
body’s  energy  generation  system.  As  a  result,  the  entire 
prosthetic  hand  actuation  structure  (inputs,  power,  strength, 
kinematics,  etc.)  must  fit  in  a  significantly  reduced  volume 
compared  to  the  human  hand  that  it  is  replacing. 

To  address  some  of  the  challenges  described  above,  this 
research  implements  a  unique  perspective  of  the  FDS  and 
FDP  muscles’  strength  space  in  the  human  forearm  and 
proposes  a  novel  design  and  parallel  actuation  structure  that 
complements  this  perspective.  The  goal  is  to  create  a  direct 
relation  between  the  forearm’s  EMG  signals  and  the 
actuation  system,  in  order  to  help  the  user  achieve  a  good 
approximation  of  the  full  capabilities  associated  with  the 
human  hand  in  a  compact  design. 


in 


Fig  1 :  Joint/Bone  composition  of  human  hand  [2] 
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Sections  2  and  3  of  this  paper  describe  the  FDS  and 
FDP  muscles’  strength  space,  how  it  relates  to  the  human 
hand’s  capabilities,  as  well  as  the  actuators  and  actuation 
structures  of  current  prosthetic/robotic  hands.  Sections  4 
and  5  will  provide  a  description  of  the  mechanical  design 
and  testing  results  that  justify  the  design’s  ability  to  execute 
the  strength  space  perspective  developed  in  this  paper.  The 
paper  will  finally  present  conclusions  associated  with  the 
testing  results  and  an  identification  of  future  work.  The 
actuation  structure’s  mechanical  design  presented  here  is  an 
expansion  of  the  content  being  published  at  the  ASME 
IDETC  2010  conference  with  the  inclusion  of  friction  in  the 
force  calculations,  implementation  of  the  mechatronics,  and 
the  EMG  inputs  being  an  expansion  of  that  presented 
research. 

The  FDS  and  FDP  muscles  are  the  primary  flexor 
muscles  in  the  human  finger  and  are  primarily  opposed  by 
the  extensor  digitorum  (ED)  muscle.  As  shown  in  Fig  2. 
The  FDP  muscle  is  attached  to  the  distal  phalanx  and  is 
capable  of  full  hand  closure;  it  is  considered  to  be  the  more 
active  of  both  finger  flexion  muscles.  The  FDS  muscle  is 
attached  to  the  middle  phalanx  and  its  full  capacity  is 
primarily  achieved  when  activation  of  the  DIP  joint  is  not 
required  or  when  full  hand  closure  tasks  require  additional 
strength  [3]. 

The  strength  space  of  the  FDS  and  FDP  muscles  is 
shown  in  Fig  3.  The  figure  demonstrates  the  normalized 
maximal  force  exertion  of  the  FDS  and  FDP  muscles  (y- 
axes)  during  maximum  force  execution  of  the  hand  (x-axes). 
The  FDP  muscle  is  shown  to  reach  its  maximal  force 
execution  (120N  [5])  at  approximately  35%  of  the  total 
flexural  effort;  however,  the  FDS  muscle  continues  to  exert 
force  until  it  reaches  its  maximal  force  execution  (240  N  [5]) 
at  about  100%  of  the  total  flexural  effort.  The  FDS  and  FDP 
behavior  can  be  attributed  to  the  learned  neurological 
activation  of  these  muscles  [6]  as  well  as  the  finger’s 
associated  kinematic  structure. 


Fig  2:  Graphic  of  FDS  and  FDP  muscle  in  finger  [4] 


Fig  3:  Strength  space  of  FDS  and  FDP  muscles  (x-axis:  normalized  postion 
of  hand  from  open  (0)  to  closed  (1),  y-axis:  normalized  force)[7] 


The  FDS  and  FDP  strength  space  comprises  the  strength 
requirements  for  all  the  tasks  that  a  hand  must  execute 
[8] [9].  In  general,  most  everyday  tasks  don’t  require 
extensive  force  but  do  require  a  certain  amount  of  dexterity. 
For  manipulation  or  approaching  an  object,  the  finger  can 
employ  both  the  FDS  and  FDP  muscles  to  nimbly 
accommodate  various  shapes  and  execute  both  general  and 
complex  movement  paths.  Fig  3  shows  that  the  FDS  and 
FDP  muscles  are  both  active  for  activities  below  35%  of  the 
maximum  force  capacity  of  the  hand  and  are  most  likely 
employed  during  manipulation/object  approaching 
movements. 

Gripping  tasks  generally  require  less  dexterity  and  more 
strength  than  manipulation  and  general  object  approach. 
The  size  and  weight  of  the  object  as  well  as  the 
characteristics  of  the  grasp  (e.g.  friction  between  pads  or 
force  closure)  determines  how  much  force  is  required  from 
the  finger’s  strength  space  shown  in  Fig  3.  It  is  to  be  noted 
that  the  actual  act  of  the  grip  also  generally  requires  much 
less  complex  finger  motion  than  that  of  manipulation. 
Based  on  these  observations,  we  divided  the  FDS  and  FDP 
strength  space  into  the  regions  shown  in  Fig  4.  In  Fig  4 
region  1  is  populated  by  the  more  frequent  dexterous  tasks 
and  region  2  is  populated  by  the  less  frequent  and  more 
strength-based  tasks. 


Fig  4:  Divided  Strength  Space  of  FDS  and  FDP  muscles  (x-axis: 
normalized  hand  position  from  open  (0)  to  closed  (1),  y-axis:  normalized 
force) 


Though  the  bones,  joints,  and  muscles  of  the  thumb  are 
somewhat  different  than  that  of  the  finger,  the  same  FDS 
and  FDP  strength  space  division  philosophy  is  applied  in  the 
mechanical  design  of  the  prosthetic  thumb. 

II.  Actuation  Structures 

To  the  authors’  knowledge,  all  currently  developed 
prosthetic/robotic  hands  use  a  single  actuator  type  to  execute 
all  the  tasks  embodied  in  the  FDS  and  FDP  strength  space. 
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This  technique  results  in  the  shortcomings  of  the  chosen 
actuator  being  carried  throughout  the  strength  space,  let  it  be 
pneumatic,  electromechanical,  ultrasonic,  or  shape  memory 
alloy.  This  could  include  excessive  size  and  weight,  or 
reduced  time  response  and  energy  inefficiency  to  name  a 
few. 

The  actuator  types  used  in  current  robotic/prosthetic 
hands  and  considered  in  this  design  included 
electromagnetic  [10][11],  pneumatic  [12] [13],  hydraulic 
[14],  ultrasonic  [15],  and  shape  memory  alloy  [16]. 
Although  all  the  listed  actuator  types  have  been  employed  in 
multiple  prosthetic  hand  designs  based  on  their 
advantageous  features,  one  or  more  shortcoming  (weight, 
noise,  size,  efficiency,  and  speed)  have  been  accepted  as 
well. 

Prosthetic  hands  have  employed  the  above  actuator  types 
into  two  general  types  of  kinematic  structures.  These  two 
structures  are  referred  to  as  fully  actuated  and 
underactuated.  The  underactuated  structure  often  uses  a 
single  input  to  actuate  the  multiple  joints  and  essentially 
wraps  the  phalanges  of  the  finger  around  an  object  [17]. 
Typically  this  is  achieved  by  a  flexible  tendon  routed 
through  the  finger  structure  which  allows  one  of  the  finger’s 
phalanges  to  be  stopped  by  the  object  without  preventing  the 
remaining  phalanges  from  continuing  to  wrap  around  the 
target 

Some  underactuated  structures  couple  all  three  degrees 
of  freedom  associated  with  finger  flexure  to  one  actuator. 
However,  there  are  structures  where  the  underactuation 
exists  only  between  the  PIP  and  DIP  joints.  This  behavior  is 
typically  executed  by  the  human  hand  and  will  be  referred  to 
herein  as  nearly  fully  actuated.  One  example  of  this  type  of 
nearly  fully  actuated  structure  is  that  adopted  by  Yamano, 
Takemura  and  Maeno  [15].  Dollar  and  Howe  present  many 
other  types  of  couplings  that  have  been  employed  in  various 
underactuated  hands  [18].  The  primary  disadvantage  of 
these  structures  is  the  lack  of  manipulation  capabilities. 

Fully  actuated  or  nearly  fully  actuated  structures  do 
allow  the  greater  manipulability  lacking  in  underactuated 
structures.  The  consequence  of  this  flexibility  is  the 
increased  number  of  motors  required  to  actuate  these 
degrees  of  freedom,  where  each  motor  must  also  be  of 
adequate  size  to  apply  the  required  forces.  This  in  turn 
increases  the  size,  weight,  and  control  complexity  of  the 
prosthetic  hand. 

Few  prosthetic  hands  are  actually  fully  actuated.  This 
could  arise  from  the  complexity  that  is  introduced  in  the 
design  or  from  the  fact  that  the  tasks  which  are  to  be 
performed  are  modeled  after  the  underactuated  human  hand 
for  grasping  actions  only.  One  hand  that  does  fully  actuate 
the  finger’s  degrees  of  freedom  is  the  UB-3  hand  [19].  In 
this  finger  each  phalange  has  a  tendon  attached  to  it  and  is 
able  to  actuate  all  flexing  degrees  of  freedom  independently. 

In  the  design  presented  in  this  paper,  the  actuators  and 
actuation  structure  were  chosen  to  specifically  complement 
the  task  characteristics  of  each  region  shown  in  Fig  4.  The 
smaller,  faster,  and  efficient  electromagnetic  motors  are 
incorporated  into  a  nearly  fully  actuated  kinematic  structure 


and  chosen  to  perform  the  tasks  associated  with  region  1. 
The  quiet,  lightweight,  strong  shape  memory  actuators 
implemented  in  a  parallel  underactuated  structure  were 
selected  to  provide  sufficient  strength  to  the  system  when 
required.  The  complementary  function  of  both  actuation 
systems  provides  the  prosthetic  hand  with  a  broad  capability 
for  grasping  and  manipulating  actions  while  trying  to 
optimize  actuator  size  and  performance. 

III.  Prosthetic  Hand  Design 

The  developed  design  is  shown  in  Fig  5.  The  design  is 
dimensionally  consistent  with  that  of  an  average  male 
human  hand  [2]  and  possesses  the  same  degrees  of  freedom. 
The  anthropomorphic  aspect  of  the  hand  is  intended  to 
enhance  the  amputee’s  acceptance  and  usability.  The  DIP 
and  PIP  joints  of  the  finger  and  the  IP  and  MCP  joints  of  the 
thumb  are  coupled.  This  is  achieved  by  connecting  a  single 
actuator  to  both  the  PIP  joint  (bevel  gears)  and  DIP  joint 
(pulley  connection  on  metacarpal  phalange).  This  coupling 
technique  is  common  among  many  prosthetic/robotic  hands 
as  noted  in  the  section  above. 


Fig  5:  Prosthetic  hand  design 


The  developed  parallel  actuation  structures  discussed  in 
the  previous  section  are  shown  in  Fig  6  and  Fig  7  for  the 
finger  and  Fig  8  and  Fig  9  for  the  thumb.  The  movements 
associated  with  region  1  in  Fig  4  are  achieved  by  two  DC 
motors.  The  DC  motors  actuating  the  coupled  DIP/PIP 
joints  of  the  finger  and  IP/MCP  joints  of  the  thumb  are 
embedded  in  the  proximal  phalange  of  the  finger  and  the 
metacarpal  phalange  of  the  thumb.  The  DC  motor  in  the 
metacarpal  phalange  of  the  finger  actuates  the  horizontal 
degree  of  freedom  of  the  MCP  joint.  The  DC  motor  at  the 
base  of  the  thumb  actuates  the  CMC  joint  to  obtain  an 
approximation  of  the  abduction/adduction  motion.  The 
second  degree  of  freedom  of  the  finger’s  MCP  joint 
(abduction/adduction)  is  only  subject  to  compliance  without 
actuation.  The  second  degree  of  freedom  in  the  thumb’s 
CMC  joint  (flexion/extension)  is  actuated  by  the  region  2 
actuation  structure. 
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Fig  6:  Region  1  actuation  scheme  for  the  finger 


Section  Actuating 
MCP  Joint 


Shape^Mernory  A  Hoy 


Fig  7:  Region  2  actuation  scheme  for  the  finger 


Fig  8:  Region  1  actuation  scheme  for  the  thumb 


CMC 


\P/P\P 
Coupling 


IP/PIP 
DC  Motor 


Fig  9:  Region  2  actuation  scheme  for  the  thumb 


The  actuation  structure  corresponding  to  region  2  in  Fig 
4  for  the  finger  includes  a  light  cable  that  passes  over  two 
restraining  shafts  in  the  MCP  joint  of  the  finger,  coils  in  the 
proximal  phalange,  and  embeds  in  the  middle  phalange.  The 


string  is  kept  in  light  tension  by  a  tension  unit  at  the  back  of 
the  hand  while  the  region  1  actuation  structure  is  active. 
When  region  2  actuation  is  required  the  shape  memory  alloy 
actuates  a  spring  loaded  cam  which  in  turn  pinches  the  string 
between  itself  and  a  roller  beneath  it.  As  the  shape  memory 
alloy  continues  to  actuate,  the  cam  introduces  the  additional 
force  required  for  region  2  tasks.  At  task  completion  the 
electric  signal  causing  the  shape  memory  alloy  to  heat  up  is 
stopped  and  the  DC  motors  and  cam  spring  extend  the  shape 
memory  alloy  back  to  its  original  state.  The  spring  loaded 
cam  mechanism  is  more  definitively  shown  in  Fig  10  and 
Fig  11. 


Fig  10:  Region  2  spring  loaded  cam  mechanism  prior  to  SMA  actuation 
(thinner  line  representing  cable  corresponds  to  lower  tension  applied  by 
tensioner) 


Fig  11:  Region  2  spring  loaded  cam  mechanism  during  SMA  actuation 
(thicker  line  representing  cable  corresponds  to  additional  tension  applied  by 
SMA  via  cam  mechanism) 


The  thumb’s  region  2  actuation  structure  is  similar  to  that 
of  the  finger’s  region  2  actuation  structure.  However,  unlike 
the  finger,  this  structure  actuates  the  degree  of  freedom  at 
the  CMC  joint  that  is  not  actuated  by  the  DC  motor.  This  is 
based  on  the  observation  that  this  degree  of  freedom  is  more 
apposing  of  the  fingers  during  tasks  that  would  require 
additional  force  (power  grasp,  high  force  pinch  grasp,  lateral 
grasp,  etc.  [8]). 
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The  design  shown  in  this  section  has  been  manufactured 
using  a  rapid  prototyping  machine.  The  prototype  can  be 
seen  in  the  Results  and  Discussion  section. 

IV.  Controller  Details 

The  control  of  the  DC  motors  in  this  research  was 
accomplished  through  the  use  of  pulse  width  modulation 
(PWM)  and  direction  control.  PWM  allowed  the  applied 
voltage  to  be  varied  continuously  which  controlled  the  speed 
and  torque  of  the  motor.  Direction  control  was  used  to 
determine  the  spin  direction  of  the  motor. 

PWM  and  direction  control  functionality  was  provided 
by  a  Pololu  Qik  2sl2vl0.  The  Qik  motor  controller 
provides  two  channels  of  speed  and  direction  control  for 
brushed  DC  motors  and  is  controlled  by  a  serial  interface.  In 
this  research,  the  motor  controller  was  connected  to  a  PC 
running  Lab  VIEW  to  provide  the  serial  commands.  The 
connection  from  the  PC  to  the  Qik  was  made  through  an 
intermediate  device,  the  Pololu  Jrk  21v3,  to  provide  the 
conversion  from  USB  to  the  required  serial  connection.  The 
Jrk  also  provides  motor  control  functionality,  but  it  was  not 
implemented  in  this  research. 

Additionally,  EMG  signal  capturing  capability  was 
added  to  control  the  DC  motors.  The  raw  signals  from  the 
EMG  sensor  are  shown  in  Fig  12  and  Fig  13. 


The  raw  signals  were  processed  by  taking  the  maximum 
absolute  value  over  a  100  sample  interval  and  generating  a 
new  data  set.  In  order  to  get  the  full  flex  and  extend  ability 
of  the  finger  from  this  new  data  set  two  threshold  levels 
where  introduced.  Above  one  threshold  the  motor  would 
spin  in  one  direction  (flex)  and  below  another  the  motor 
would  spin  in  the  opposite  direction  (extend).  The  deadzone 
was  implimented  as  a  buffer  area  to  transition  between  the 
flexation  and  extention  zones. 


The  EMG  signal  was  acquired  from  the  first  author’s 
digit  III  FDP  muscle  and  supplied  as  an  analog  voltage  from 
a  separate  PC  incorporating  a  Delsys  Bagnoli  EMG  system. 
The  analog  voltage  was  sampled  through  a  National 
Instruments  data  acquisition  card  installed  in  the  motor 
control  PC.  The  analog  signal  was  then  filtered  and  scaled 
using  Lab  VIEW  to  a  value  representing  the  amplitude  of  the 
EMG  signal,  which  was  then  used  to  control  the  direction  of 
the  motors.  A  sample  of  the  filtered  and  scaled  signal  with 
thresholds  is  shown  in  Fig  14. 


The  entire  controller  setup  can  be  seen  in  the  schematic 
shown  below  in  Fig  15. 


Motor  0  Motor  1 

(MCP)  (DIP/PIP) 

Fig  15:  Motor  controller  setup 


V.  Experimental  Setup 

The  experiment  associated  with  this  research  was 
performed  to  validate  the  designed  actuation  structure’s 
ability  to  span  a  two-region  strength  space  similar  to  the  one 
identified  above  for  the  human  hand.  The  actuators  used  in 
the  prototype  were  Pololu  298:1  Micro  Metal  Gearmotor  HP 
DC  motors  [21]  and  the  Electric  Piston  SMA  actuator  from 
Raychem  [22].  These  actuators  are  relatively  inexpensive 
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and  the  implementation  of  more  expensive  actuators  could 
further  enhance  the  values  reported  below. 

The  experiment  consisted  of  having  the  finger  and  the 
thumb  grip  a  FSR  sensor  fastened  to  a  dense  foam  ball  in  the 
large  grip  and  then  the  close  grip  configurations  as  shown  in 
Fig  16  through  Fig  19.  The  large  grip  setup  simulated  the 
fmger/thumb  performing  a  more  robust  grasp  on  a  larger 
object  and  the  small  grip  setup  simulated  the  actuation 
structure  of  the  fmger/thumb  grasping  smaller  objects  or 
performing  the  more  dexterous  pinch  or  lateral  grasp.  The 
values  obtained  experimentally  are  compared  to  the 
expected  calculated  values  using  a  simple  friction  inclusive 
static  calculation  of  the  reaction  force  required  at  the  FSR 
sensor  to  resist  the  stall  torque  of  the  DC  motors  and  the 
measured  tension  provided  by  the  SMA. 


Fig  16:  Finger  in  large  grip  configuration. 


Fig  17:  Finger  in  close  grip  configuration. 


Fig  18:  Thumb  in  close  grip  configuration. 


Fig  19:  Thumb  in  large  grip  configuration. 


The  experimental  values  were  compared  to  the  expected 
ones  using  friction  inclusive  static  calculations  based  on  the 
free  body  and  geometric  diagrams  shown  in  Fig  20  and  Fig 
21.  The  “T”  vectors  in  the  free  body  diagrams  represent  the 
tension  in  the  cable.  The  tension  values  are  calculated  using 
(1)  where  the  Vj  and  Vj  are  the  geometric  vectors  of  the  cable 
on  either  side  of  a  pivot  point,  tL  and  tj  are  the  tension  values 
in  these  vectors,  and  p  is  the  friction  coefficient  of  the  pivot 
point.  Because  tj  is  the  only  unknown  in  this  equation  it  is 
separated  and  solved  for  in  order  to  determine  the  next 
tension  vector  in  the  system. 

|  CD 


Fig  20:  Primary  variables  used  in  friction  inclusive  static  calculations  of 
finger  in  both  close  and  large  grip  configurations. 


M2b 


Fig  21 :  Primary  variables  used  in  friction  inclusive  static  calculations  of 
thumb  in  both  close  and  large  grip  configurations. 
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With  the  tension  vectors  determined,  the  simple  static 
equations  (1)  and  (2)  for  the  finger  and  thumb  are  used  to 
solve  for  the  forces/torques  of  each  joint  from  the  distal 
phalange  to  the  knuckle  in  terms  of  the  variable  F  value  then 
using  the  equations  to  solve  for  F.  The  stall  torque  applied 
by  the  DC  motors  (90  oz-in)  replaced  the  Ml  and  M2a/M2b 
variables.  As  described  in  the  design  section  M2a  and  M2b 
are  coupled  and  their  relationship  is  shown  in  (3).  The 
gravitational  forces  generated  by  each  phalange  was  applied 
at  the  center  of  the  link  in  the  downward  direction  using  the 
mass  values  for  the  finger  of  (Mprox=21.6g,  Mmid=9.45g, 
Mdist=6.7g)  and  for  the  thumb  of  (Mmet=33.3g,  Mprox=10.5g, 
Mdist=8.5g). 


2>= o 

(2) 

TF=0 

(3) 

M2a  +  —  M2b  =  90oz  -  in  (4) 
14 


VI.  Results  and  Discussion 

Each  of  the  four  configuration  setups  were  performed 
three  times  with  each  actuator  type  individually  and  then 
with  them  combined.  The  results  are  shown  in  Table  1 
where  the  columns  correspond  to  the  resulting  forces  from 
the  SMA,  DC  motors,  the  results  of  the  SMA  and  DC 
motors  column  summed  together,  the  experienced  force 
when  the  two  actuator  types  are  physically  applied  at  the 
same  time,  and  the  value  calculated  using  basic  statics.  The 
table  shows  that  when  applied  individually,  the  resulting 
forces  from  the  SMA  are  for  the  most  part  larger  than  the 
applied  DC  motor  forces  for  all  four  configurations.  The 
average  combined  forces  for  the  finger  are  generally  greater 
than  the  experienced  but  the  values  for  the  thumb  are 
approximately  the  same. 

Table  1 :  Finger  &  Thumb  Testing  Results 


Finger  Ml  &  M2  &  SMA  (Newtons) 


Finger  CG 

SMA 

DC  Motors 

Combined 

(Sum) 

Combined 

(Experimental) 

Combined5 

(Calculated) 

Combined1' 

(Calculated) 

Average 

7.75 

9.78 

17.52 

15.23 

16.55 

16.85 

Std  Dev 

0.21 

0.22 

0.15 

0.20 

0.08 

0.08 

1  1 

Finger  LG 

SMA 

DC  Motors 

Combined 

(Sum) 

Combined 

(Experimental) 

Combined5 

(Calculated) 

Combined1" 

(Calculated) 

Average 

8.17 

8.67 

16.84 

15.08 

15.23 

15.46 

Std  Dev 

1.26 

1.14 

1.27 

0.80 

0.30 

0.30 

Thumb  Ml  &  SMA  (Newtons) 


Thumb  CG 

SMA 

DC  Motor 

Combined 

(Sum) 

Combined 

(Experimental) 

Combined5 

(Calculated) 

Combined1" 

(Calculated) 

Average 

4.70 

1.24 

5.94 

5.60 

5.14* 

5.61* 

Std  Dev 

0.90 

0.35 

0.86 

1.14 

0.05 

0.06 

1  1 

Thumb  LG 

SMA 

DC  Motor 

Combined 

(Sum) 

Combined 

(Experimental) 

Combined5 

(Calculated) 

Combined1" 

(Calculated) 

Average 

4.56 

3.45 

8.01 

8.18 

7.70* 

8.27* 

Std  Dev 

1.85 

0.45 

2.45 

0.39 

0.49 

0.58 

*  Values  combined  from  calculated  SMA  values  and  measured  DC  Motor  values 
§  Values  where  calculated  using  friction  coefficients  of  p(nylon)=0.102  and  p(steel)=0.09 
£  Values  where  calculated  using  friction  coefficients  of  p(nylon)=0.52  and  gi(steel)=0.09 


The  finger  behavior  is  expected  because  the  SMA  was 
only  partly  isolated  from  the  motors  as  the  motors  were  still 
employed  during  the  SMA  test  in  order  to  maintain  the 
structure  of  the  finger  against  the  ball  prior  to  testing.  This 
resulted  in  the  SMA  having  to  press  against  the  ball  and 
overcome  the  minor  resistance  presented  by  the  motors.  The 
thumb  behavior  was  expected  as  well  due  to  the  slightly 
compliant  units  isolating  the  motor  during  the  motor  test 
absorbing  some  of  the  applied  motor  force.  The  calculated 
forces  where  also  expectedly  higher  than  the  measured  data 
as  friction  from  the  system  was  not  incorporated  in  their 
development. 

It  is  to  be  noted  that  the  combined  (calculated)  values  for 
the  thumb  are  only  calculated  for  the  SMA  input  and  the  DC 
Motor  experimental  value  has  been  added  to  the  calculated 
value.  This  was  done  because  a  part  of  the  component  in  the 
prototype  thumb  broke  causing  the  gearing  to  not  mesh 
appropriately,  giving  significantly  smaller  force  values 
(approx.  3N)  than  those  calculated  (approx.  14N). 

The  controller  was  only  applied  to  the  DC  motors  of 
finger  and  thumb  in  their  close  grip  configurations.  The 
objective  of  these  tests  where  to  merely  validate  that  the 
controller  platform  had  the  power  and  functional  capacity  to 
produce  similar  results  to  that  of  the  power  supply  inputs 
using  an  EMG  signal  input. 

The  EMG  voltage  and  current  outputs  for  both  the  finger 
and  thumb  were  consistent  with  that  of  the  direct  power 
supply  inputs.  The  EMG  thumb  test  resulted  in  a  DC  motor 
force  of  0.95N  which  agreed  with  the  1.24N  +  0.35N 
generated  by  the  direct  power  supply  tests.  It  was  also 
visually  verified  that  the  finger  squeezed  the  ball  to  a  similar 
degree  with  the  EMG  controller  as  the  direct  power  supply 
test  system.  However,  a  later  identified  disconnect  in  the 
FSR  sensor  did  not  allow  the  generation  of  reliable  force 
data  for  the  finger.  Successive  experiments  for  the  finger 
resulted  in  a  break  in  the  rapid  prototyped  proximal 
phalange.  This  experiment  is  being  repeated  immediately 
with  a  soon  to  be  re-built  proximal  phalange 

VII.  Conclusions  and  Future  Work 

This  paper  presents  a  novel  design  and  actuation  system 
for  a  prosthetic  hand.  The  actuation  structure  was  shown  to 
effectively  span  a  two-region  strength  space  to  execute  grip 
configurations  similar  to  those  found  in  the  three  primary 
grips  of  the  human  hand  (power,  pinch,  and  lateral).  The 
design  also  showed  the  ability  of  an  underactuated  and 
nearly  fully  actuated  kinematic  structure  to  exist  in  a  single 
actuation  system  using  remarkably  different  types  of 
actuators,  without  compromising  the  required  size  and 
weight  of  the  prosthetic  hand. 

According  to  these  results,  the  parallel  actuation  structure 
is  a  good  starting  point  for  the  design.  The  results  also 
validated  that  the  design  could  be  effectively  driven  by  an 
EMG  signal.  Complete  testing  of  the  parallel  actuation 
system’s  performance  will  require  the  expansion  of  the 
EMG  inputs  to  perform  different  grasps  and  manipulations. 

Future  work  includes  further  testing,  development  of  the 
controller,  fabrication  of  a  prototype  that  is  more  robust  yet 
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of  the  same  form  as  the  rapid  prototype  components  tested 
here,  and  the  use  of  the  dynamical  model  of  the  hand  for 
manipulation  tasks,  in  order  to  improve  the  design.  More 
precisely  selected  actuators  will  be  used  in  the  final  design. 
The  stronger  actuators  will  add  the  appropriate  scale  to  the 
developed  strength  space  so  as  to  mimic  the  force  generation 
capabilities  of  the  system  as  well  as  the  strength  space  form. 
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Abstract:  Recent  evidence  suggests  si  I  icon  dioxide  micro-  and  nanoparticles  induce  cytotoxic 
effects  on  lung  cells.  Thus,  there  is  an  increasing  concern  regarding  their  potential  health 
hazard.  Nevertheless,  the  putative  toxicity  of  nanoparticles  in  mammalian  cel  Is  has  not  yet  been 
systematically  investigated.  We  previously  noted  that  several  metallic  oxide  nanoparticles  exert 
differential  cytotoxic  effects  on  human  neural  and  nonneural  cel  Is.  Therefore,  we  hypothesized 
that  silicon  dioxide  nanoparticles  induce  cytotoxicity  in  U87  cells  by  lowering  their  survival 
by  decreasing  cell  survival  signaling  and  disturbing  mitochondrial  function.  To  investigate  this 
hypothesis,  we  determined  the  activities  of  the  key  mitochondrial  enzymes,  citrate  synthase  and 
mal ate  dehydrogenase,  in  astrocytoma  U 87  cells  treated  with  silicon  dioxide  nanoparticles.  In 
addition,  we  studied  the  expression  of  the  mitochondrial  DNA-encoded  proteins,  cytochrome 
C  oxidase  II  and  nicotinamide  adenine  dinucleotide  (NAD PH)  dehydrogenase  subunit  6,  and 
cell  signaling  pathway  protein  extracellular  signal-regulated  kinase(ERK)  and  phosphorylated 
ERK  in  treated  U87  cells.  The  activated  form  of  ERK  controls  cell  growth,  differentiation, 
and  proliferation.  In  parallel,  we  determined  survival  of  U87  cells  after  treating  them  with 
various  concentrations  of  silicon  dioxide  nanoparticles.  Our  results  indicated  that  treatment 
with  silicon  dioxide  nanoparticles  induced  decreases  in  U87  cell  survival  in  a  dose-related 
manner.  The  activities  of  citrate  synthase  and  malate  dehydrogenase  in  treated  U87  cells 
were  increased,  possibly  due  to  an  energetic  compensation  in  surviving  cells.  However,  the 
expression  of  mitochondrial  DNA-encoded  cytochrome  C  oxidase  subunit  II  and  NADH 
dehydrogenase  subunit  6  and  the  cell  signaling  protein  ERK  and  phosphorylated  ERK  were 
altered  i  n  the  treated  U  87  cells,  suggesting  that  silicon  dioxide  nanoparticles  induced  disruption 
of  mitochondrial  DNA-encoded  protein  expression,  leading  to  decreased  mitochondrial  energy 
production  and  decreased  cell  survival/proliferation  signaling.  Thus,  our  results  strongly 
suggest  that  the  cytotoxicity  of  silicon  dioxide  nanoparticles  in  human  neural  cells  implicates 
altered  mitochondrial  function  and  cell  survival/proliferation  signaling. 

Keywords:  cytotoxicity,  silicon  dioxide  nanoparticles,  mitochondrial  enzyme,  extracellular 
signaling  regulated  kinase,  cell  signaling,  neural  cells 


Correspondence:  Solomon  W  Leung 
Department  of  Civil  and  Environmental 
Engineering,  College  of  Engineering  and 
Biomedical  Research  Institute,  Idaho  State 
University,  Pocatello,  ID,  USA 
Email  leunsolo@isu.edu 


Introduction 

According  to  the  Environmental  Protection  Agency,  exposure  to  nanomaterials  can 
occur  during  the  manufacturing  or  production  process,  and  also  has  the  potential  to 
pollute  the  environment,1  in  this  case  the  occupational  environment.  A  case  in  point 
is  exposure  to  silicon  dioxide  nanoparticles  because  silicon  dioxide,  including  a 
range  of  its  particle  sizes,  is  being  used  in  cosmetics,  food,2  and  drug  formulations.3 
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In  general,  silicon  dioxide  is  viewed  as  a  nontoxic  substance 
and  its  industrial  applications  are  numerous.2-4  Nevertheless, 
there  have  been  reports,  at  least  in  the  last  two  decades,  that 
silicon  dioxide  particles  are  not  as  harmless  as  they  were 
previously  assumed  to  be. 

W  hen  the  size  of  silicon  dioxide  is  above  the  micrometer 
scale  (ie,  >1  pm),  it  appears  to  be  benign  to  human,  insects, 
and  microorganisms.  However,  diatomaceous  earth,  which 
contains  more  than  85%  of  amorphous  silicon  dioxide,  is 
used  as  insecticide  in  dust  form,4  but  its  toxicity  to  insects 
is  not  well  defined.  When  the  size  of  silicon  dioxide  is  in 
the  micrometer  scale  and  below,  several  toxic  effects  have 
been  observed  in  humans  and  animals.  Most  noticeable  is 
the  disease  of  silicosis  in  humans,5  resulting  from  prolonged 
exposure  to  crystal  I  i  ne  si  I  ica  dust,  and  si  mi  I  ar  symptoms  have 
also  been  reported  in  rats.5 Thus  far,  silicon  dioxide  toxicity 
reports  have  been  almost  exclusively  on  particle  sizes  at  the 
micrometer  scale,  and  mostly  concerning  pulmonary  injuries 
or  lung  inflammation.5'6  Few  studies  have  been  conducted  on 
silicon  dioxide  at  the  nanometer  scale,56  and  even  fewer  stud¬ 
ies  so  far  reported  have  been  conducted  with  human  cells. 

In  macroscopic  sizes,  silicon  dioxide  is  not  known  to  be 
cytotoxic.2'3'5'6  However,  a  recent  study  demonstrated  that 
exposure  of  amorphous  spherical  silicon  dioxide  nanoparticles 
of  different  sizes  induced  decreases  in  viability  of  human 
endothelial  cells,  an  expression  of  their  cytotoxicity  which  was 
apparently  dependenton  their  particle  size.7  N  evertheless,  few, 
if  any,  studies  have  examined  the  putative  cytotoxic  effects  of 
silicon  dioxide  nanoparticles  on  human  neural  cells. 

T  here  was  an  early  study  i  ndi  cati  ng  that  si  I  i  ca  parti  cl  es  ( i  n 
the  micrometer  and  nanometer  ranges)  when  introduced  into 
the  brains  of  rats  and  mice  induced  an  inflammatory  response 
in  brain  astrocytes  and  macrophages,  and  the  degeneration 
of  some  adjacent  axons  and  axon  terminals,  as  elucidated 
by  light  and  electron  microscopy.8  Furthermore,  silicon  and 
al  umi  num  were  found  to  be  co-l  ocal  ized  i  n  the  central  regi  on 
of  senile  plaque  cores  in  the  cortex  of  patients  with  senile 
dementi  a  of  the  A  Izhei  mer  type,9  and  the  accumul  ated  si  I  icon 
and  aluminum  appeared  to  be  localized,  at  least  in  part,  in 
lipofuscin  granules  in  the  brains  of  patients  who  died  with 
Alzheimer's  disease.10  Consequently,  these  early  studies8-10 
suggest  that  silicon  dioxide  nanoparticles  may  exert  cytotoxic 
effects  on  neural  cells. 

We  recently  found  that  exposure  to  titanium  dioxide  and 
two  other  metallic  oxide  (namely,  ZnO  and  MgO)  nano¬ 
particles  induced  differential  cytotoxicity  on  human  neural 
cells.11  Because  of  our  recent  findings11  and  the  increasing 
evidence  that  a  variety  of  nanoparticles  (including  silicon 


dioxide-containing  nanoparticles)  can  cross  the  blood-brain 
barrier,1213  we  initiated  this  study  to  investigate  the  putative 
cytotoxic  effect(s)  of  silicon  dioxide  nanoparticles  on  human 
neural  cells.  In  particular,  we  focused  on  investigation  of 
the  putative  cytotoxic  effects  of  these  nanoparticles  on 
mitochondrial  function  and  survival  cell  signaling  in  human 
astrocytoma  U87  cells  that  are  astrocyte- 1  ike  in  view  of 
the  fact  that  they  are  good  models  of  astrocytes  normally 
found  in  the  brain  in  vivo14'15  and  that  astrocytes  in  the  brain 
are  functionally  coupled  to  endothelial  cells  and  hence  the 
functions  of  the  blood- brain  barrier.16'17 

Because  of  the  aforementioned  considerations,  we 
hypothesized  that  silicon  dioxide  nanoparticles  induce 
cytotoxicity  in  U87  cells  by  disturbing  mitochondrial  function 
and  by  lowering  cell  survival  via  decreasing  cell  survival 
signaling.  To  investigate  our  hypothesis,  we  determined 
the  effects  of  treatment  with  various  concentrations  of 
silicon  dioxide  nanoparticles  on  the  activities  of  two  key 
mitochondrial  enzymes,  ie,  citrate  synthase  and  malate 
dehydrogenase,  and  expression  of  the  mitochondrial  DNA- 
encoded  protei  ns,  cytochrome  oxidase  subunit  1 1  and  N  A  D  H 
dehydrogenase  subunit  6,  andthecell  survival  signaling  path¬ 
way  protein  ERK  and  phosphorylated  ERK  in  U 87  cells. 

Materials  and  methods 

Materials 

Silicon  dioxide  nanoparticles  (size  12  nm)  were  purchased 
from  STREM  Chemicals,  Newburyport,  MA.  Dulbecco’s 
modified  Eagle's  medium  (DM  EM )  and  other  chemicals  were 
purchased  from  Sigma-Aldrich  (St.  Louis,  M  0).  U87  cells 
were  obtained  from  American  Type  Culture  Collection 
(M  anassas,  VA). 

Cell  culture 

Human  U87  astrocytoma  cells  were  cultured  using  DM  EM 
supplemented  with  10%  (v/v)  fetal  bovine  serum  and 
antimycotic  solution.  The  cells  were  maintained  in  a  75  cm2 
flask  containing  20  mL  medium,  incubated  at37°C  in  aC02 
incubator.  When  the  cells  reached  70%-80%  confluence 
they  were  harvested  either  by  scrapping  or  trypsinization 
and  transferred  to  other  flasks. 

Preparation  of  stock  suspension 
of  silicon  dioxide  nanoparticles 

The  silicon  dioxide  nanoparticles  stock  suspension  was 
first  prepared  by  suspending  50  mg  of  silicon  dioxide 
nanoparticles  in  a  100  mL  conical  flask  containing  sterile 
phosphate-buffered  saline.  The  suspension  was  then  placed 
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in  a  magnetic  stirrer  to  allow  themixtureto  stir  continuously 
at  room  temperature  before  further  dilutions  were  made 
from  the  stock  (which  was  being  continuously  stirred)  to  be 
used  at  specified  concentrations  and  then  mixed  for  use  in 
treating  cells. 

MTT  assay  to  assess  cell  viability 

C  el  I  s  were  seeded  ( 2000  cel  I  s/wel  I )  onto  a  96-wel  I  pi  ate  and 
allowed  to  attach  to  the  bottom  of  each  well  (inner  diameter 
0.6  cm)  for  an  hour.  Silicon  dioxide  nanoparticles  were 
then  added  at  specified  concentrations  (0.1-100  pg/mL)  to 
each  well  containing  0.2  mL  with  the  cells  attached  at  the 
bottom  and  mixed.  The  plates  so  prepared  were  incubated 
for  48  hours  at  37°C.  MTT  dye  was  then  added  to  each  well 
and  the  plate  incubated  for  another  four  hours  at  37°C.  The 
purple  crystals  formed  in  cel  Is  were  dissolved  using  dimethyl 
sulfoxide  and  the  absorbance  of  the  resultant  material  in  the 
wells  was  measured  at  567  nm  using  the  microplate  reader 
as  described  previously.11 

Determination  of  mitochondrial 
enzyme  activity 

The  U87  cells  were  cultured  in  DM  EM  in  75  cm2  flasks. 
When  the  cells  were  70%-80%  confluent,  silicon  dioxide 
nanoparticle  treatments  were  initiated.  The  mitochondrial 
function  of  the  untreated  and  treated  U  87  cells  was  assessed 
by  measuring  activities  of  two  tricarboxylic  acid  cycle 
enzymes,  citrate  synthase  and  malate  dehydrogenase,  in 
U87  cells  treated  with  silicon  dioxide  nanoparticles  at  0, 25, 
50,  or  100  pg/mL  by  previously  published  methods.18  The 
untreated  U87  cells  served  as  the  control.  Citrate  synthase 
and  malate  dehydrogenase  activity  in  cell  homogenates  was 
measured  at  412  nm  and  340  nm,  respectively,  using  an 
ultraviolet-V IS  spectrophotometer.18 

Western  blot  analysis 

Expression  of  the  mitochondrial  DNA-encoded  proteins, 
NADPH  dehydrogenase  subunit  6  and  cytochrome  C 
oxidase  subunit  II,  and  the  cell  signaling  proteins  ERK 
and  phosphorylated  ERK  were  determined  by  Western  blot 
analysis.  The  U  87  cells  treated  with  or  without  (ie,  control) 
silicon  dioxide  nanoparticles  werecollected  and  homogenized 
in  a  buffer  containing  5  mM  HEPES  and  250  mM  sucrose, 
along  with  protease  inhibitors  as  described  previously.15  Pro¬ 
tein  content  of  the  homogenates  was  then  determined  using 
the  bicinchoninic  acid  technique  with  a  microplate  reader.11 
B  riefly,  equal  amounts  of  protein  from  the  samples  were  loaded 
onto  the  lanes  of  the  gels,  and  the  proteins  were  separated 


by  polyacrylamide  gel  electrophoresis  and  transferred  to  a 
polyvinylidene fluoride  membrane.19  M  onoclonal  antibodies 
against  the  respective  proteins  were  then  used  to  probe  the 
proteins  of  interest.  The  polyvinylidene  fluoride  membrane 
containing  the  target  protein  was  then  developed  using  the 
chemiluminescence  technique  on  an  X-ray  film  to  assess  the 
extent  of  expression  of  respective  proteins.19 

Assessment  of  cellular  morphology 

The  morphology  of  the  U87  cells,  treated  with  silicon 
dioxide  nanoparticles  for  48  hours  at  37°C  using  specified 
concentrations,  were  compared  with  that  of  the  control  or 
untreated  U87  cells  by  using  bright  field  light  microscopy. 
The  images  were  obtained  using  a  Leica  light  microscope  at 
400  x  magnification  (L eica  D M  I R B ;  Leica,  B annockburn,  I L ) 
equipped  with  a  digital  camera  (Leica  DFC  300FX;  Leica).15 

Statistical  analysis 

Data  analysis  was  performed  employing  one  way  A  NOVA 
followed  by  Dunnett's  post  hoc  test  using  SPSS  software 
(SPSS  Inc.,  Chicago,  I L).  A II  data  reported  aremean  +  SEM 
of  three  individual  experiments.  Statistical  significance  was 
set  at  P  <  0.05. 

Results 

Effect  of  nanoparticles  on  human  U87 
astrocytoma  cell  survival 

To  determine  the  effect  of  silicon  dioxide  nanoparticles 
on  cell  survival,  U87  cells  were  exposed  to  silicon  dioxide 
nanoparti  cl  es  for  48  hours  at  concentrations  rangi  ng  from  0. 1 
to  100  pg/mL.  At  lower  treatment  concentrations,  from  0.1 
to  10  pg/mL,  the  nanoparticles  did  not  affect  viability  of  the 
U  87  cells  (Figure  1).  H  owever,  at  treatment  concentrations  of 
25  pg/mL  and  higher,  silicon  dioxide  nanoparticles  induced 
concentration-related  decreases  in  survival  of  U87  cells. 


MTT  assay 


Figure  I  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  survival  of  human 
astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon 
dioxide  nanoparticles  for  48  hours.  Values  were  the  mean  ±  SEM  of  at  least  three 
separate  experiments;  CTRL  represented  the  untreated  (ie,  control)  U87  cells; 
*P  <  0.05  versus  control  cells. 
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At  the  highest  treatment  level  of  100  pg  /mL,  less  than  30% 
of  the  cells  survived  (Figure  1). 

Effect  on  mitochondrial  function 
in  human  U87  astrocytoma  cells 

Because  cell  survival  critically  depends  on  mitochondrial 
functions  being  maintained  at  a  normal  physiologic  level, 
we  determined  the  effect  of  silicon  dioxide  nanoparticles 
on  mitochondrial  function  in  U87  cells  by  monitoring  the 
activities  of  citrate  synthase  and  malate  dehydrogenase.13 
Both  enzymes  are  nuclear  DNA-encoded;  these  enzyme 
proteins  are  synthesized  in  the  endoplasmic  reticulum  and 
then  imported  into  the  mitochondrial  matrix  compartment. 

At  treatment  concentrations  of  25-100  pg/mL  for 
48  hours,  silicon  dioxide  nanoparticles  induced  dose-related 
i  nc  reases  i  n  c  i  trate  sy  nthase  acti  vi  ti  es  i  n  U  8  7  cel  I  s  ( F  i  g  u  re  2 ) . 
On  the  other  hand,  although  at  the  same  concentrations  the 
nanoparticles  also  induced  significantly  increased  activity 
in  malate  dehydrogenase  in  U87  cells,  the  increases  were 
not  dose- related  (Figure  3).  Using  the  same  nanoparticle 
concentrations  for  treatment  of  U87  cells,  there  was  a  dose- 
related  decrease  in  cell  survival  (Figure  1),  and  it  is  likely 
that  the  remai  ni  ng  survi  vi  ng  U  87  cel  I  s  were  compensati  ng  by 
upregulation  of  citrate  synthase  and,  to  a  less  extent,  malate 
dehydrogenase,  so  as  to  maintain  their  energy  production  via 
tricarboxylic  acid  cycle  metabolism  for  survival. 

Effects  of  nanoparticles  on  mitochondrial 
DNA-encoded  and  cell  signaling  protein 
expression 

Because  silicon  dioxide  nanoparticles  induced  dose-related 
decreases  in  survival  of  U87  cells  at  concentrations  of 
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Figure  2  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  specific  activities 
of  citrate  synthase  in  human  astrocytoma  U87  cells.  U87  cells  were  treated  at 
specified  concentrations  of  silicon  dioxide  nanoparticles  for  48  hours.  Then  the 
activities  of  citrate  synthase  in  the  homogenates  of  treated  and  untreated  (ie, 
control,  Ctrl)  U87  cells  were  determined  as  described  in  Materials  and  methods; 
the  activities  of  citrate  synthase  were  expressed  per  mg  of  homogenate  protein  as 
specific  activities.  The  specific  activities  values  were  the  mean  ±  SEM  of  at  least  three 
separate  experiments;  Ctrl  represented  the  value  in  untreated  U87  cell  homogenate; 
*P  <  0.05  versus  that  of  control  cells. 
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Figure  3  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  specific  activities 
of  malate  dehydrogenase  in  human  astrocytoma  U87  cells.  U87  cells  were  treated 
at  specified  concentrations  of  silicon  dioxide  nanoparticles  for  48  hours.  Then  the 
activities  of  malate  dehydrogenase  in  the  homogenates  of  treated  and  untreated  (ie, 
control,  Ctrl)  U87  cells  were  determined  as  described  in  Materials  and  methods;  the 
activities  of  malate  dehydrogenase  were  expressed  per  mg  of  homogenate  protein  as 
specific  activities.  The  specific  activity  values  were  the  mean  ±  SEM  of  at  least  three 
separate  experiments;  Ctrl  represented  the  value  in  untreated  U87  cell  homogenate; 
*P  <  0.05  versus  that  of  control  cells. 


25-100  |ag  /mL  over  48  hours  (Figure  1),  we  investigated  the 
possibility  that  these  decreases  in  survival  can  be  attributed 
to  the  nanoparticle-induced  alterations  in  expression  of 
mitochondrial  DNA-encoded  and  cell  signaling  proteins 
(Figures  4-6). 

At  treatment  concentrations  of  25-100  pg/mL  for 
48  hours,  silicon  dioxide  nanoparticles  induced  dose-related 
decreases  i  n  the  expression  of  cytochrome  C  oxidase  subunit 
II  and  NADH  dehydrogenase  subunit  6  (two  mitochondrial 
DNA-encoded  peptides)  in  U87  cells  (Figures  4  and  5). 
These  results  strongly  suggested  treatment  with  silicon 
dioxide  nanoparticles  altered  the  normal  functioning  of  the 
mitochondrial  genome  in  U87  cells.  A  likely  consequence 
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Figure  4  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of 
cytochrome  oxidase  subunit  II  in  human  astrocytoma  U87  cells.  U87  cells  were 
treated  at  specified  concentrations  of  silicon  dioxide  nanoparticles  for  48  hours. 
Then  cell  lysates  of  treated  and  untreated  (ie,  control)  U87  cells  were  prepared 
as  described  in  Materials  and  methods.  The  expression  of  cytochrome  oxidase 
subunit  II,  encoded  by  mitochondrial  DNA,  was  determined  by  Western  blot 
analysis  using  P-actin  as  the  loading  control:  Lane  I,  lysate  of  untreated  or  control 
U87  cells;  lane  2,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at 
25  pg/mL;  lane  3,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles  at 
50  pg/mL;  lane  4,  lysate  of  U87  cells  treated  with  silicon  dioxide  nanoparticles 
at  100  pg/mL.  The  blots  were  from  a  typical  experiment.  Two  other  experiments 
yielded  essentially  the  same  trend  of  results. 
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Figure  5  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of 
nicotinamide  adenine  dinucleotide  (NADH)  dehydrogenase  subunit  6  in  human 
astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of  silicon 
dioxide  nanoparticles  for  48  hours.  Then  cell  lysates  of  treated  and  untreated 
(ie,  control)  U87  cells  were  prepared  as  described  in  Materials  and  methods.  The 
expression  of  NADH  dehydrogenase  subunit  6,  encoded  by  mitochondrial  DNA, 
was  determined  by  Western  blot  analysis  using  P-actin  as  the  loading  control:  Lane 
I,  lysate  of  untreated  or  control  U87  cells;  lane  2,  lysate  of  U87  cells  treated  with 
silicon  dioxide  nanoparticles  at  25  pg/mL;  lane  3,  lysate  ofU87  cells  treated  with  silicon 
dioxide  nanoparticles  at  50  pg/mL;  lane  4,  lysate  of  U87  cells  treated  with  silicon 
dioxide  nanoparticles  at  1 00  pg/mL.  The  blots  were  from  a  typical  experiment.  Two 
other  experiments  yielded  essentially  the  same  trend  of  results. 

of  this  effect  is  impairment  in  the  mitochondrial  electron 
transport  chain  in  U87  cells.  Indeed,  the  nanoparticle-induced 
disruption  of  mitochondrial  respiratory  chain  structure  and 
function  may  be  one  cause  of  energy  failure  that  ultimately 
led  to  the  death  of  U  87  cells. 

Si02  nano  (|xg/mL) 
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Figure  6  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  expression  of 
extracellular  signal  regulated  kinase  (ERK)  and  phosphorylated  ERK  (p-ERK)  in 
human  astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations  of 
silicon  dioxide  nanoparticles  for  48  hours.  Then  cell  lysates  of  treated  and  untreated 
(ie,  control)  U87  cells  were  prepared  as  described  in  Materials  and  methods.  The 
expression  of  ERK  and  phosphorylated  ERK  was  determined  by  Western  blot 
analysis  using  P-actin  as  the  loading  control:  Lane  marked  C,  lysate  of  untreated 
or  control  U87  cells;  lane  marked  I,  lysate  of  U87  cells  treated  with  silicon  dioxide 
nanoparticles  at  I  pg/mL;  lane  marked  10,  lysate  of  U87  cells  treated  with  silicon 
dioxide  nanoparticles  at  10  pg/mL;  lane  marked  50,  lysate  of  U87  cells  treated  with 
silicon  dioxide  nanoparticles  at  50  pg/mL.  The  blots  were  from  a  typical  experiment. 
Two  other  experiments  yielded  essentially  the  same  trend  of  results. 


Other  than  their  effect  of  depressing  the  mitochondrial 
respiratory  chain,  thereby  decreasing  oxidative  phos¬ 
phorylation,  treatment  of  U87  cells  with  silicon  dioxide 
nanoparticles  could  also  lower  survival  of  U87  cel  Is  through 
altering  cell  signaling  pathway(s)  that  regulate(s)  cell 
survival  and  proliferation.20  We  therefore  investigated  this 
possibility  by  examining  the  effect  of  these  nanoparticles 
on  expression  of  ERK  and  phosphorylated  ERK  proteins. 
Weobserved  thatwhen  U87  cells  were  treated  with  silicon 
dioxide  nanoparticles  1-50  jug/mL  for  48  hours,  the  protein 
expression  of  phosphorylated  ERK  showed  a  dose-related 
decrease,  while  that  of  ERK  protein  remained  essentially 
unchanged  (Figure  6),  suggesting  that  a  lowering  of  this  cel  I 
survival/proliferation  signaling  mechanism  could,  at  least 
in  part,  account  for  the  dose-related  decrease  in  U87  cell 
survival  induced  by  these  nanopartides. 

Effect  on  morphology  of  U87  cells 

To  ascertain  whether  or  not  treatment  of  U87  cells  with 
silicon  dioxide  nanoparticles  led  to  any  gross  changes  in 
their  morphology,  we  examined  the  cells  under  bright  field 
I  ight  microscopy  (Figure  7).  We  observed  that  as  we  exposed 
U87  cells  to  increasing  concentrations  of  silicon  dioxide 
nanoparticles  higher  than  25  jag/mL ,  the  cells  became  more 
and  more  enlarged  and  swollen,  suggesting  that  they  assumed 
a  pathologic  appearance  (Figure  7). 

Discussion 

Few,  if  any,  recent  studies  have  addressed  the  cytotoxic 
effects  of  silicon  dioxide  nanoparticles  in  neural  cells. 
As  far  as  we  are  aware,  ours  is  the  first  to  report  the 
cytotoxic  effects  of  silicon  dioxide  nanoparticles  on 
human  astrocytoma  U87  cells  and  some  of  the  underlying 
subcel I ular  mechanisms.  Consistent  with  our  hypothesis 
that  silicon  dioxide  nanoparticles  induce  cytotoxicity  in 
U87  cells  by  disturbing  mitochondrial  function  and  by 
lowering  cell  survival  viadecreasing  cell  survival  signaling, 
we  found  that  treatment  with  silicon  dioxide  nanoparticles 
induced  concentration-related  lowering  of  protein  expression 
of  the  mitochondrial  DNA-encoded  cytochrome  C  oxidase 
subunit  II  and  NAD  FI  dehydrogenase  subunit  6,  two  key 
components  of  the  mitochondrial  respiratory  chain,  and 
phosphorylated  ERK,  a  key  cell  survival/proliferation 
signaling  molecule  in  U87  cells.  It  is  noteworthy  that  we 
found  those  changes  in  protein  expression  at  concentrations 
of  silicon  dioxide  nanoparticles  that  correlated  with  those 
inducing  concentration-related  decreases  in  survival  of 
U87  cells. 
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Figure  7  Effect  of  treatment  with  silicon  dioxide  nanoparticles  on  the  morphology  of  human  astrocytoma  U87  cells.  U87  cells  were  treated  at  specified  concentrations 
of  silicon  dioxide  nanoparticles  for  48  hours.  Then  the  morphology  of  treated  and  untreated  (ie,  control)  U87  cells  were  assessed  using  bright  field  light  microscopy  at 
400  x  magnification  as  described  in  Materials  and  methods.  The  photomicrographs  shown  were  representative  of  those  obtained  in  several  studies. 


As  emphasized  earlier,  the  initial  rational e for  focusi ng 
our  studies  on  U  87  cel  Is  was  that  few  studies  have  addressed 
the  putative  cytotoxic  effects  of  silicon  dioxide  nanopar¬ 
ticles  on  neural  cells,  especially  those  from  the  human 
brain.  M  ore  importantly,  not  only  do  astrocytes  have  many 
physiologic  roles  (eg,  metabolic  trafficking,  neurotrans¬ 
mitter  cycling,  protecting  neurons  from  pathophysiologic 
assaults),15’21'22  they  also  play  pathophysiologic  roles  in 
disease  states  (eg,  neuroinflammation  in  neurodegenera- 
tive  diseases  such  as  Alzheimer's  disease  and  Parkinson's 
disease).22  Furthermore,  it  is  relevant  to  note  that  although 
U87  are  neurotumor  cells,  they  possess  many  of  the  "nor¬ 
mal"  physiologic  characteristics  of  astrocytes  detected  in 
primary  cultures  of  astrocytes  and  astrocytes  in  the  brain 
in  vivo.1415Thus,  U87  cells  constitute  a  good  model  system 
in  vitro  for  brain  astrocytes.1415  Consequently,  we  have 


employed  U87  cells  as  a  model  system  to  elucidate  the 
putative  cytotoxicity  of  nanoparticles  on  neural  cells  in  a 
systematic  manner.11 

One  neurotoxic  effect  of  a  neurotoxicant,  such  as 
manganese,  is  via  a  lowering  of  cell  survival/prolifera¬ 
tion  signaling  molecule  expression,  leading  ultimately  to 
death  of  neural  cells.20  Consistent  with  this  observation  is 
our  finding  in  this  study  that  treatment  of  U87  cells  with 
increasing  concentrations  of  silicon  dioxide  nanoparticles 
induced  a  dose-related  decrease  in  protein  expression  of 
phosphorylated  ERK  (Figure6),  one  importantcell  survival/ 
proliferation  signaling  molecule.20  This  effect  correlated 
quite  well  with  the  dose-related  decrease  in  survival  of 
these  cells  (Figure  1)  and  alterations  in  their  morphology 
(Figure  7)  induced  by  the  nanoparticles.  M  oreover,  we  also 
found  that  i  n  U  87  cel  I  s  treated  i  n  the  same  nanoparti  cl  e  dose 
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range,  there  was  a  concentration-related  decrease  in  protein 
expression  of  mitochondrial  DNA -encoded  cytochrome  C 
oxidase  subunit  II  and  NADH  dehydrogenase  subunit  6 
(Figures  4  and  5). 

Our  present  findings  that  exposure  of  human  astrocy¬ 
toma  U87  cells  to  silicon  dioxide  induced  dose-related 
decreases  in  cel  I  survival  and  alteration  of  their  mitochondrial 
structure  and  function  (Figures  1-6)  are  similar  to  those  of 
C  hen  et  al23  who  demonstrated  that  exposure  of  human  brai  n 
microvascular  endothelial  cel  Is  to  aluminum  oxide  nanopar¬ 
ticles  induced  decreased  viability  of  those  cells,  alteration 
in  their  mitochondrial  membrane  potential,  and  a  lowering 
of  their  tight  junction  protein  expression.  Consequently, 
in  this  context,  our  finding  that  silicon  dioxide  nanopar¬ 
ticles  induced  a  dose-related  lowering  of  the  expression  of 
cytochrome  C  oxidase  subunit  1 1  and  NADH  dehydrogenase 
subunit  6  is  of  pathophysiologic  importance  and  interest  in 
several  respects.  Because  these  two  peptides  are  structural 
components  of  the  mitochondrial  respiratory  chain  com¬ 
plexes  IV  and  I,  respectively,  and  have  key  functional  roles 
in  that  respiratory  chain,24  decreased  expression  of  these 
peptides  induced  by  the  nanoparticles  likely  results  in  dys¬ 
function  in  the  mitochondrial  respiratory  chain  and  oxidative 
phosphorylation,  leading  ultimately  to  decreased  adenosine 
triphosphate  synthesis.  Consequently,  the  decreased  expres¬ 
sion  of  the  two  mitochondrial  DNA-encoded  peptides  could 
likely  contribute  to  decreasing  U87  cell  survival  induced 
by  silicon  dioxide  nanoparticles  through  necrotic  cell  death 
as  a  result  of  energy  failure.11  This  conclusion  is  certainly 
consistent  with  our  observation  that  the  U  87  cells  assumed 
an  increasingly  swollen  appearance,  a  hallmark  of  necrosis, 
as  we  increased  their  treatment  levels  of  silicon  dioxide 
nanoparticles  (Figure  7).  The  decreased  expression  of  cyto¬ 
chrome  C  oxidase  subunit  II  and  NADH  dehydrogenase 
subunit  6  peptides  induced  by  the  nanoparticles  could  reflect 
the  effect  of  the  nanoparticles  on  altering  communication 
between  the  mitochondrial  genome  and  the  nuclear  genome, 
decreasing  mitochondrial  peptide  synthesis,  disruption  of 
protein  targeting  to  mitochondria,  and/or  a  combination 
of  these  mechanisms.  The  nanoparticle-induced  decrease 
in  protein  expression  of  the  two  peptides  could  also  result 
from  the  overall  effect  of  the  nanoparticles  on  survival/ 
proliferation  signaling  in  U87  cells.  Clearly,  these  are  novel 
but  mechani  sti  cal  ly  rel  evant  possi  bi  I  i ti  es  that  deserve  further 
investigation. 

Unlike  the  downregulation  of  the  protein  expression 
of  mitochondrial  DNA-encoded  cytochrome  C  oxidase 
subunit  II  and  NADH  dehydrogenase  subunit  6,  and  the 


survival/proliferation  signaling  molecule  phosphorylated 
ERK,  treatment  of  U87  cells  with  silicon  dioxide  nano¬ 
particles  induced  concentration-related  increases  in  the 
activity  of  citrate  synthase  and  malate  dehydrogenase, 
both  of  which  are  located  in  the  mitochondrial  matrix 
(Figures  2  and  3).  Because  the  increases  in  citrate  synthase 
and  malate  dehydrogenase  activity  were  observed  at 
nanoparticle  concentrations  that  also  induced  dose-related 
decreases  in  U87  cell  survival  (Figure  1),  one  possible,  and 
likely,  explanation  for  the  increased  citrate  synthase  and 
malate  dehydrogenase  activity  is  that  the  surviving  cells 
were  showing  compensatory/adaptive  mechanisms  involv¬ 
ing  upregulation  of  tricarboxylic  acid  cycle  enzymes  in  the 
mitochondrial  matrix  of  surviving  or  dying  cells,  or  both,  in 
the  face  of  downregulation  of  mitochondrial  DNA-encoded 
respiratory  chain  components,  such  as  cytochrome  C 
oxidase  subunit  II  and  NADH  dehydrogenase  subunit  6. 
Consequently,  this  upregulation  of  tricarboxylic  acid  cycle 
enzymes  may  offset  (or  compensate  for)  the  effect  on 
mitochondrial  energetics  through  the  downregulation  of 
respiratory  chain  components  induced  by  the  nanoparticles. 
Only  future  studies  can  elucidate  whether  this  mechanistic 
possibility  is  tenable  because  they  are  beyond  the  scope  of 
the  present  study. 

Because  ours  is  the  first  study  to  report  on  the  cytotoxic 
effects  of  silicon  dioxide  nanoparticles  on  human  neural 
cells,  only  a  few  comparisons  can  be  made  with  the  effects 
of  silicon  dioxide  nanoparticles  on  neural  and  peripheral 
cell  types  because  the  literature  in  this  area  of  research  is 
rather  scant.11  For  example,  Napierska  et  al7  demonstrated 
exposure  of  human  endothel  i  al  cel  I  s  to  amorphous  spheri  cal 
silicon  dioxide  nanoparticles  induced  decreases  in  their 
viability  and  this  effect  was  apparently  dependent  on  their 
particle  size.  Several  groups  had  also  shown  that  silicon 
dioxide  nanoparticles  decreased  viability  of  some  peripheral 
human  and  other  mammalian  cell  types.5'25'26  Thus,  recent 
reports,5'7'25'26  as  well  as  the  findings  of  this  study,  are  in 
accord  in  emphasizing  that  silicon  dioxide  nanoparticles 
induces  decreased  viability  of  a  variety  of  human  and  other 
mammalian  cell  types.  Nevertheless,  more  systematic  stud¬ 
ies  are  required  to  elucidate  fully  the  molecular  mechanisms 
underlying  the  cytotoxicity  of  these  nanoparticles. 

As  we  discussed  previously,  one  early  study  indicated 
that  when  silicon  nanoparticles  were  introduced  directly  into 
the  brains  of  rats  and  mice,  they  induced  an  inflammatory 
response  in  brain  astrocytes.8  Furthermore,  silicon  was 
found  in  senile  plaque  cores  and  in  lipofuscin  granules  in 
the  cortex  of  patients  who  died  of  Alzheimer's  disease.910 
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In  view  of  these  earlier  findings,8-10  because  astrocytes 
play  important  roles  in  neuroinflammation,22  and  U87  cells 
constitute  a  good  cell  model  for  astrocytes,  the  results 
of  this  study  may  have  pathophysiologic  implications  in 
the  inflammation  in  astrocytes  in  Alzheimer's  disease  in 
particular  and  in  neuroinflammation  in  general.  Thus,  the 
mechanistic  connection  between  the  inflammatory  response 
in  Alzheimer's  disease,  neuroinflammation,  and  the  cytotoxic 
effects  of  silicon  dioxide  nanoparticles  in  neural  cells  merits 
further  investigation. 

Conclusion 

This  study  is  the  first  to  report  on  the  cytotoxic  effects 
of  silicon  dioxide  nanoparticles  on  human  astrocytoma 
U87  cells,  and  some  of  the  underlying  subcellular 
mechanisms.  Consistent  with  our  hypothesis,  we  found 
that  treatment  with  silicon  dioxide  nanoparticles  induced 
concentration-related  lowering  of  protein  expression  in 
the  mitochondrial  DNA-encoded  cytochrome  C  oxidase 
subunit  II  and  NADH  dehydrogenase  subunit  6,  two  key 
components  of  the  mitochondrial  respiratory  chain,  and 
phosphorylated  ERK,  a  key  cell  survival/proliferation  sig¬ 
naling  molecule  in  U87  cells.  We  also  noted  those  changes 
in  protein  expression  induced  by  silicon  dioxide  nanopar¬ 
ticles  correlated  with  concentration-related  decreases  in 
survival  of  U87  cells  induced  by  the  nanoparticles.  On  the 
other  hand,  we  observed  upregulation  of  citrate  synthase 
and  malate  dehydrogenase,  two  key  tricarboxylic  acid 
cycle  enzymes.  Those  changes  were  likely  compensatory/ 
adaptive  mechanism  of  dying  and/or  surviving  U87  cells. 
Thus,  our  findings  may  assume  pathophysiologic  impor¬ 
tance  in  neuroinflammation  and  Alzheimer's  disease  and 
in  the  environmental  health  impact  of  human  exposure  to 
these  nanoparticles.  This  is  obviously  a  worthwhile  area 
for  further  study. 
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Abstract : 

Diabetic  neuropathy  is  one  of  the  most  severe  complications  of  chronic  diabetes. 
Though  several  theories  exist,  the  pathophysiology  of  the  disease  is  poorly  understood. 
We  have  therefore  developed  an  in  vitro  model  to  elucidate  its  underlying 
pathophysiologic  mechanisms.  We  believe  our  model  allows  us  to  gain  new  insights  into 
such  mechanisms.  Moreover,  our  model  improves  on  existing  primary  culture  models 
because  the  latter  change  with  passage  number  and  have  limited  life-span.  We 
hypothesize  that  high  glucose  treatment  induces  oxidative  stress  and  decreases  GSH 
level,  ultimately  leading  to  apoptosis  in  dorsal  root  ganglion  (DRG)  neurons  and 
Schwann  cells.  Results  of  our  ongoing  studies  suggest  exposure  of  DRG  neurons  to 
high  levels  of  glucose  (e.g.,  45  &  60  mM),  conditions  that  favor  increased  ROS 
generation,  decrease  their  survival,  eventually  leading  to  cell  death  whereas  in  co¬ 
culture,  Schwann  cells  help  protect  DRG  neurons  and  enhance  their  survival.  Other 
studies  are  in  progress  to  further  characterize  our  cell  model.  Thus,  results  of  our 
ongoing  studies  are  consistent  with  the  notion  that  our  cell  model  is  appropriate  for 
investigation  of  diabetic  neuropathy. 

Keywords:  Cell  Culture  Model,  Diabetic  Neuropathy,  Neurodegeneration 

Introduction: 

Diabetes  is  the  disease  of  high  blood  glucose  and  according  to  the  American 
Diabetic  Association,  23.6  million  people  (8%  of  United  States  populations)  have 
diabetes.  Diabetic  neuropathy  is  one  of  the  most  severe  complications  of  diabetes,  an 
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inflammatory  condition  in  which  the  nerves  in  the  extremities  of  the  body  become 
damaged,  ultimately  resulting  in  death  of  nerve  cells  (Brownlee  2001,  Figueroa-Romero 
et  al.  2008,  Leinninger  et  al.  2006).  It  affects  more  that  60%  of  diabetic  population 
(Figueroa-Romero  et  al.  2008).  Some  of  the  serious  complications  of  nerve  damage  as 
a  result  of  diabetes  include  loss  of  sensation  or  pain  in  limbs,  adverse  effects  on 
digestion,  decreased  sexual  response,  and  urinary  tract  infection.  Half  of  the  foot 
amputations  in  the  United  States  constitute  the  consequence  of  diabetic  neuropathy. 
There  is  little  advancement  in  the  management  and  treatment  of  diabetic  neuropathy 
because  it  is  very  difficult  to  predict  and  monitor  nerve  cell  damage. 

Various  in  vivo  and  in  vitro  models  mimicking  closely  or  approximately  the 
diabetic  pathologies  in  humans  have  been  devised  to  investigate  pathophysiological 
mechanisms  of  the  disease.  Each  model  has  its  own  advantages  and  limitations.  In  vivo 
models  are  preferred  in  elucidating  the  etiology  of  the  disease  and  for  testing  the 
efficacy  of  potential  therapies.  However,  limitations  of  in  vivo  models  include  inability  to 
tightly  control  their  glycemic  status  and  the  fact  that  these  models  do  not  lend 
themselves  readily  for  use  in  investigating  the  pathophysiology  of  the  disease  at  the 
cellular  and  molecular  levels  (Apfel  2006). 

Unlike  in  vivo  models,  in  vitro  models  of  diabetes  in  general  and  diabetic 
neuropathy  in  particular  allow  investigations  of  pathophysiological/pathogenic 
mechanisms  at  the  cellular  and  molecular  levels  (Sango  et  al.  2006).  For  example, 
these  models  permit  the  examination  of  effects  of  high  glucose  on  neurons  in  a 
controlled  environment  (Sango  et  al.  2006).  Nevertheless,  there  is  no  in  vitro  cell  model 
available  whereby  one  could  study  the  long-term  exposure  of  dorsal  root  ganglion 


Page  417  of  531 


(DRG)  neurons  and/or  Schwann  cells  to  high  and  pathologic  levels  of  glucose.  To 
address  this  need  for  such  model(s),  we  have  been  systematically  developing  different 
co-culture  models  of  DRG  neurons  and  Schwann  cells  because  the  DRG  neurons  in 
vivo  are  always  surrounded  by  Schwann  cells  and  Schwann  cells  protect  them  from  any 
damage  or  cell  death  (Vincent  et  al.  2005).  We  hypothesize  that  immortalized  cell 
culture  models  using  DRG  neurons,  Schwann  cells  and  their  co-cultures  can  be 
productively  employed  to  elucidate  the  pathophysiological  and/or  pathogenic 
mechanisms  of  diabetic  neuropathy.  This  study  was  initiated  to  investigate  our 
hypothesis. 

Materials  and  Methods: 

Cell  Cultures.  The  immortalized  DRG  neurons  were  a  gift  from  Dr.  Ahmet  Hoke’s 
laboratory  at  Johns  Hopkins  University  (Baltimore,  MD,  USA)  (Chen  et  al.  2007).  The  rat 
Schwann  cell  line  was  obtained  from  ATCC  (Manassas,  VA,  USA).  These  cells  and  their 
co-culture  were  cultured  in  DMEM  (Sigma;  St  Louis,  MO,  USA)  supplemented  with  10% 
(v/v)  fetal  bovine  serum  (FBS),  1%  (w/v)  sodium  pyruvate,  0.292  g/l  L-glutamine,  1.5  g/l 
sodium  bicarbonate,  1%  (v/v)  anti-mycotic  and  various  specified  concentrations  (5-45 
mM)  of  glucose. 

Cell  Survival  (MTT)  Assay.  8000-10000  DRG  neurons  or  Schwann  cells  were  seeded 
per  well  in  a  48-well  plate  in  a  medium  with  a  normal  or  high  glucose  concentration. 
After  incubation  at  37°C  for  48  hours,  50  pi  of  MTT  (5  mg/ml  in  PBS)  reagent  was  added 
to  each  well.  After  incubation  for  another  4  hours,  the  medium  was  removed  gently  and 
the  cellular  reaction  product  was  extracted  in  200  pi  DMSO.  Then  the  optical  density  of 
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contents  of  each  well  was  measured  in  a  plate  reader  at  570  nm.  The  absorbance  was 
proportional  to  the  number  of  live  cells  previously  present  in  each  well. 

Reactive  Oxygen  Species  (ROS)  Assay.  Cells  in  a  medium  with  normal  or  high 
glucose  were  incubated  at  37°C  for  48  hours  and  then  200  pi  of  H2DCF  dye  (10  pM  in 
PBS)  was  added  to  each  well.  After  another  45  minutes,  the  medium  was  removed 
gently  and  200  pi  of  PBS  was  added  in  each  well.  The  fluorescence  output  in  each  well 
was  measured  in  the  plate  reader  set  at  489  nm  excitation-521  nm  emission  in  the 
fluorescence  mode.  The  fluorescence  output  corresponded  to  the  ROS  produced  by 
cells  present  in  each  well  and  was  normalized  with  respect  to  cellular  protein,  which  was 
determined  using  the  BCA  Protein  Assay  Kit. 

Glutathione  (GSH)  Assay.  GSH  in  cell  homogenates  was  measured  using  Ellman’s 
reagent  (5,5'-dithiobis-(2-nitrobenzoic  acid)  or  DTNB)  as  described  previously 
(Dukhande  et  al.  2006). 

Co-culture  of  DRG  neurons  and  Schwann  cells.  Our  co-culture  model  closely  mimic 
the  physiological  conditions  of  DRG  neurons  &  Schwann  cells  because  Schwann  cells 
always  surround  DRG  neurons  in  vivo  and  protect  them  from  pathophysiological 
assaults.  DRG  neurons  &  Schwann  cells  were  cultured  at  37°C  and  5%  CO2  in 
Dulbecco’s  Modified  Eagle’s  Medium  supplemented  with  10%  FBS  and  counted  using  a 
Coulter  counter  after  24  hours  or  at  about  60-70  %  confluence.  DRG  neurons  were 
seeded  on  a  Corning  Costar  24-well  plate.  Schwann  cells  were  then  seeded  on  a 
Millipore  hanging  cell  culture  insert  (pore  size  0.4  pm)  on  a  separate  plate  and  then 
introduced  to  a  DRG  neuron-containing  well  after  4  hours  to  achieve  the  co-culture 
condition.  DRG  neurons  (co-cultured  with  Schwann  cells)  were  exposed  to  various 
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levels  of  glucose  for  specified  periods  and  their  survival  determined  employing  the  cell 
survival  assay. 

Statistical  analysis  of  results.  Statistical  significance  of  experimental  results  was 
analyzed  with  one-way  ANOVA  followed  by  Dunnett’s  post-hoc  test  using  SPSS  17 
software  package,  with  a  minimum  significance  level  set  at  p<  0.05. 

Results  and  Discussion: 

Diabetic  neuropathy  affects  the  peripheral  nervous  system  (PNS)  the  most: 
dorsal  root  ganglion  (DRG)  neurons  in  PNS  are  especially  susceptible  to  high  glucose- 
induced  cell  damage  and/or  death.  Therefore,  we  used  DRG  neurons  to  develop  a  cell 
culture  model  for  diabetic  neuropathy  and  have  begun  to  characterize  it.  When  we 
exposed  DRG  neurons  to  various  glucose  levels,  we  noted  that  these  cells  survived 
best  at  25  mM  of  glucose  (Fig.  1),  a  level  higher  than  the  physiological  level  of  5.5  mM. 
Relevant  to  our  finding  (Fig.  1)  is  the  observation  of  Izawa  et  al.  (2009)  who  found  that 
lactate  produced  by  Schwann  cells  may  be  a  fuel  for  DRG  neuronal  energy  metabolism. 
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Fig.  1  Survival  of  DRG  Neurons  after  24-hour  Exposure  to  Different  Glucose 
Concentrations. 
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Fig.  2  Survival  of  DRG  Neurons  after  exposure  to  high  glucose  for  24  hours. 

(*p<0.05  compare  to  25  mM  of  glucose) 

Schwann  cells  usually  surround  DRG  neurons  in  vivo,  provide  them  with 
glycolytic  intermediates  (Miller  et  al.  2002)  and  protect  them  from  pathophysiological 
assaults.  Consistent  with  this  notion  is  our  observation  that  both  cell  types  survived 
better  in  co-culture  than  when  they  were  cultured  alone  (data  not  shown).  Furthermore, 
when  DRG  neurons  are  cultured  alone,  they  need  more  glucose  because  of  the 
absence  of  the  glycolytic  intermediates  normally  supplied  to  them  by  Schwann  cells 
(Miller  et  al.  2002).  Based  on  the  above  considerations  and  our  finding  that  DRG 
neurons  showed  optimal  survival  with  25  mM  glucose  (Figs.  1  and  2),  we  employed  25 
mM  of  glucose  as  a  control  parameter  when  DRG  neurons  and  Schwann  cells  are 
cultured  separately  and  5.5  mM  of  glucose  as  a  physiological  control  when  we  co¬ 
cultured  them. 

Exposure  of  neural  cells  to  high  glucose  increases  reactive  oxygen  species  (ROS) 
production,  a  by-product  of  glucose  metabolism  and  marker  of  diabetic  neuropathy 
(Vincent  et  al.  2005).  Consistent  with  this  notion  is  our  finding  that  ROS  production  by 
DRG  neurons  was  increased  whey  they  were  exposed  to  glucose  level  higher  than  25 
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mM  (Fig.  3).  Clearly  the  mechanisms  underlying  the  increased  ROS  production  by  DRG 
neurons  in  high  glucose  merit  further  elucidation. 
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Fig.  3  ROS  Production  by  DRG  Neurons  after  Exposures  to  High  Glucose 
Concentrations 
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Fig.  4  Changes  in  Glutathione  Level  in  Schwann  Cells  after  Exposure  to  High 
Glucose  for  24  hr  (*p<0.05  compare  to  25  mM  of  glucose) 

Because  glial  cells  in  brain  have  a  protective  role  on  neurons  (Dukhande  et  al. 

2006),  we  investigated  the  possibility  that  under  the  stress  of  exposure  to  high  glucose, 
Schwann  cells  may  produce  less  glutathione  (antioxidant)  thereby  decreasing  their 


Page  422  of  531 


ability  to  protect  DRG  neurons.  We  exposed  Schwann  cells  to  high  glucose  and 
measured  their  glutathione  content.  Consistent  with  this  hypothesis  is  our  finding  that 
exposure  to  high  glucose  induced  decreases  in  glutathione  in  Schwann  cells  compared 
to  their  levels  when  treated  with  25  mM  of  glucose  (Fig.  4). 

In  conclusion,  results  from  this  study  strongly  suggest  our  model  is  suitable  for 
systematic  investigation  of  mechanisms  underlying  pathophysiology  and  pathogenesis 
of  diabetic  neuropathy.  Thus,  these  results  (and  those  to  be  obtained  in  our  ongoing 
studies)  provide  additional  insights  into  cellular  and  molecular  mechanisms  relevant  to 
diabetic  neuropathy  and  as  such  may  be  implicated  not  only  in  this  disease  but  also  in 
other  chronic  diseases  such  as  Alzheimer’s  disease  and  Parkinson’s  disease. 
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Further  Elucidation  of  Neuroprotective  Properties  of  Astrocytoma  (Astrocytes-like) 
Cells 

Abstract: 

Astrocytes  protect  neurons  against  various  assaults  although  the  underlying 
pathophysiological  and  molecular  mechanisms  are  poorly  understood.  Therefore,  we 
have  continued  to  develop  cell  models  in  vitro  to  further  investigate  the  putative 
neuroprotective  properties  of  astrocyte-like  astrocytoma  cells.  We  hypothesized  that 
different  stress  factors  can  elicit  and/or  enhance  the  neuroprotective  effects  of 
astrocytes  through  the  activation  of  astrocytic  survival  signaling  and  alteration  of 
astroglial  function.  Employing  cell  models  involving  astrocytes-like  U87  astrocytoma  and 
neurons-like  SK-N-SH  cells,  results  of  our  previous  and  ongoing  studies  suggest  that 
U87  cells  protect  SK-N-SH  cells  against  cytotoxicity  due  to  oxidative  stress.  Other 
ongoing  studies  target  the  putative  neuroprotective  effect(s)  of  U87  cells  due  to 
alterations  of  cell  signaling  and/or  other  astroglial  function(s).  Thus,  our  results  may 
have  implications  for  the  pathophysiology  of  neuroprotection. 

Keywords:  Astrocytes;  neuroprotection;  astroglial  function;  U87  astrocytes-like 
astrocytoma  cells;  SK-N-SH  neurons-like  neuroblastoma  cells 

Introduction: 

Astrocytes  (non-neuronal,  glial  cells)  in  the  brain  are  known  to  play  important 
roles  in  protecting  neurons  (nerve  cells)  against  various  assaults  although  the 
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underlying  mechanisms  are  incompletely  understood  (Sofroniew  et  al.  2010).  Well 
known  chronic  neurodegenerative  diseases,  such  as  Alzheimer’s  disease  (AD)  and 
Parkinson’s  disease  (PD),  are  correlated  with  neuronal  cell  death,  which  involves  many 
factors  alone  or  in  combination  (Przedborski  et  al.  2003).  Among  all  neurodegenerative 
diseases,  significant  attention  has  been  given  to  Alzheimer’s  disease  (AD),  following  by 
Parkinson’s  disease  (PD)  (Przedborski  et  al.  2003).  While  the  signs  and  symptoms  of 
AD  affect  such  patients’  quality  of  life,  the  conditions  that  lead  to  memory  loss  and  death 
of  neurons  in  this  disease  state  are  not  yet  defined.  AD  remains  one  of  the  top  ten 
leading  causes  of  death  in  the  elderly  population  in  the  United  States  and  it  is  referred 
as  the  progressive  and  irreversible  neurodegenerative  disorder  in  human  brain  (Chung, 
2009).  Current  therapy  predominantly  treats  AD  symptoms  and,  at  best,  helps  the 
disease  from  worsening  as  there  is  no  known  cure  available  for  AD.  Hence,  there  is  an 
urgent  need  to  better  understand  how  astrocytes  exert  their  protective  effects  on 
neurons.  Once  the  mechanisms  for  astrocyte-mediated  neuronal  protection  are 
elucidated,  one  can  design  treatment  strategies  to  prevent  the  occurrence  of 
accelerated  neuronal  loss  in  neurodegenerative  diseases  such  as  AD  and  PD. 

Our  project  seeks  to  examine  the  underlying  neuroprotective  properties  offered 
by  astrocytes  to  protect  neurons  employing  a  cell  model  system  in  vitro.  We  have 
therefore  developed  cell  model  systems  to  investigate  cytotoxic  and  inflammatory 
mechanisms  associated  with  induction  of  neuronal  cell  damage  and  cell  death 
(Malthankar  et  al.  2004,  Dukhande  et  al.  2006,  Wang  et  al.  2008).  We  have  previously 
shown  that  U87  astrocytoma  and  SK-N-SH  neuroblastoma  cells  constitute  good  model 
systems  for  astrocytes  and  neurons,  respectively,  in  vitro  because  of  their  close 
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functional  resemblance  to  normal  astrocytes  and  neurons  and  that  U87  cells  protect  SK- 
N-SH  cells  from  several  pathological  assaults  (Malthankar  et  al.  2004,  Dukhande  et  al. 
2006).  We  therefore  hypothesized  that  different  stress  factors  (e.g.  oxidative  stress  and 
nutrient  depletion)  can  elicit  and/or  enhance  the  neuroprotective  effects  of  astrocytes 
through  the  activation  of  astrocytic  signaling  and  alteration  of  astroglial  function.  In  this 
study,  we  have  investigated  this  hypothesis  further. 

Materials  and  Methods: 

Cell  culture 

The  human  astrocytoma  (U87)  and  neuroblastoma  (SK-N-SH)  cell  lines  were 
obtained  from  (ATCC;  Manassas,  VA,  USA)  and  cultured  in  Minimum  Essential  Medium 
(MEM)  supplemented  with  10%  (v/v)  fetal  bovine  serum  (Atlanta  Biologicals; 
Lawrenceville,  GA),  1%  (w/v)  sodium  pyruvate  (Sigma  Aldrich;  St  Louis,  MO),  0.292  g/L 
L-glutamine  (Sigma),  1.5g/L  sodium  bicarbonate  (Sigma)  and  1%  (v/v)  antimycotic 
(Atlanta  Biologicals).  Cells  were  maintained  at  37°C  and  5%  (v/v)  CO2  as  described 
previously  (Malthankar  et  al.  2004,  Dukhande  et  al.  2006). 

Treatment  of  U87  cells 

The  U87  cells  were  cultured  in  T-75  tissue  culture  flasks  until  ~50%  confluent  in 
20  ml  of  10%  FBS  MEM  medium.  Then  the  medium  was  discarded  and  changed  with 
20  ml  of  5%  FBS-MEM  medium  and  the  cells  incubated  for  24  hours.  After  that,  the 
medium  was  replaced  with  20  ml  of  medium  without  FBS  and  the  cells  incubated  for  an 
additional  24  hours.  Cells  were  treated  with  5mM  L-sulfoximine  buthionine  (BSO),  or 
20nM  pioglitazone  (PZ)  or  a  combination  of  both  frugs  at  the  same  concentrations.  At 
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the  end  of  another  24  hours  of  incubation,  the  media  were  first  removed  and  frozen  with 
liquid  nitrogen  immediately.  The  cells  were  collected  subsequently  and  both  cells  and 
media  collected  were  kept  at  -80°C  until  used  for  experiments. 

Cell  Survival  (MTT)  assay 

The  neuroprotective  properties  of  U87  cells  on  SK-N-SH  cells  were  determined  in 
different  combinations  of  regular  media  (RM)  and  conditioned  media  (CM)  collected 
from  U87  cells  cultured  (see  above)  with  RC/CM  ratios  of  100/0,  70/30,  50/50,  40/60, 
30/70,  and  0/100.  5000  SK-N-SH  cells  were  seeded  per  well  in  a  48-well  plate.  After 
48  hours  of  incubation  at  37°C,  50  pi  thiazolyl  blue  tetrazolium  bromide  (MTT)  (5  mg/ml 
in  PBS)  reagent  was  added  to  each  well.  The  medium  in  each  well  was  removed  after  4 
hours  at  37°C  and  the  purple  formazan  crystals  formed  were  solubilized  using  DMSO. 
Optical  densities  (OD)  of  the  contents  of  the  wells  were  read  in  a  plate  reader  at  570  nm; 
the  OD  corresponded  to  the  proportion  of  live  cells  in  each  well  (Dukhande  et  al.  2006). 
Western  Blot  Analysis 

Western  blot  analysis  was  used  to  assess  protein  expression  in  cell  lysates  to 
determine  the  effects  of  media,  BSO  and  PZ.  10%  SDS-PAGE  gel  electrophoresis  was 
performed  as  described  by  Dukhande  et  al.  (2006).  The  separated  proteins  were  probed 
with  anti-glutamine  synthetase  and  anti-glial  fibrillary  acidic  protein  antibodies  and  then 
visualized  using  chemiluminesence  detection  kit  with  (3-actin  as  the  loading  control. 
Statistical  Analysis  of  Data 

Statistical  significance  of  experimental  results  was  analyzed  with  one-way 
ANOVA  followed  by  Tukey’s  post-hoc  test  with  a  minimum  significance  level  set  at  p  < 
0.05  using  Kaleidagraph  4.0  software  package. 
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Results  and  Discussion: 


Being  a  selective  inhibitor  of  glutathione  (a  naturally-occurring  antioxidant) 
synthesis,  L-buthionine  sulfoximine  (BSO)  is  used  to  induce  oxidative  stress  in  cells. 
Pioglitazone  (PZ),  a  peroxisome  proliferator  activated  receptor  gamma  (PPARy)  agonist, 
is  used  to  block  the  inflammation  which  is  usually  unregulated  in  AD  (Yamagishi  2000, 
Syversen  et  al.  2009).  Consequently,  we  designed  studies  whereby  we  could  use  BSO 
and  PZ  to  further  elucidate  the  putative  protective  mechanism(s)  on  neurons-like  SK-N- 
SH  cells  derived  from  conditioned  media  in  which  astrocytes-like  U87  cells  had  been 
exposed  to  BSO  or  PZ  or  a  combination  of  both  drugs. 

Serum  depletion  and  drug  treatment  for  U87  cell  cultures  were  prepared  as 
shown  the  flow  chart  below  (Fig.  1).  U87  cells  cultured  in  medium  containing  10%  FBS 
were  used  as  control  (not  shown  in  Fig.  1).  Medium  containing  0%  FBS  were  defined 
as  serum  free  (SF)  medium. 
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Figure  1.  Experimental  design  of  U87  cells  treated  with  BSO,  PZ  or  in  combination. 

Because  oxidative  and  other  environmental  stresses  can  induce  morphological 
and  function  changes/adaption  in  astrocytes  in  vivo  and  in  culture,  we  employed  light 
microscopy  (by  taking  photomicrographs  at  lOOx  magnification)  to  examine  the  effects 
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of  BSO  treatment  and  FBS  depletion  to  induce  oxidative  and  environmental  stress, 
respectively,  on  U87  cells  and  assess  the  effects  of  PZ  treatment  thereon. 
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Figure. 2  Treatment-induced  cellular  morphology  changes  in  U87  cells  (100X  magnification). 


In  Figure  2,  the  top  panel  of  photomicrographs  shows  U87  cells  cultured  in  10% 
FBS  medium  treated  with  or  without  drug  or  drug  combination  while  those  of  the  bottom 
panel  represents  U87  cells  cultured  in  serum-deprived  (SF)  medium  treated  with  or 
without  drug  or  drug  combination.  In  general,  the  photomicrographs  revealed  that 
treatment  with  SF  medium  induced  U87  cells  to  extend  multipolar  processes.  Moreover, 
treatment  of  U87  cells  with  or  without  drug  treatment  led  to  further  changes  in  cellular 
morphology  (e.g.,  extension  of  processes  and  shape  changes)  in  addition  to  that  noted 
in  the  U87  cells  cultured  in  SF  medium.  Clearly,  these  observations  raise  interesting 
mechanistic  questions  that  need  to  be  addressed  further:  we  were  able  to  address  two 
of  these  issues  (see  below)  while  others  are  beyond  the  scope  of  this  study. 

GFAP  is  an  established  marker  for  astrocytes:  its  altered  expression  is 
associated  with  changes  in  astroglial  morphology.  Thus,  we  employed  Western  blot  to 
determine  if  the  morphological  changes  in  U87  cells  exposed  to  SF  medium  with  and 
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without  BSO/PZ  treatment  were  also  associated  with  changes  in  glial  fibrillary  acid 
protein  (GFAP)  expression.  GFAP  expression  was  increased  when  U87  cells  were 
cultured  in  SF  medium  compared  to  those  cultured  in  10%  FBS  medium,  suggesting  a 
correlation  between  changes  its  expression  and  cellular  morphology  (data  not  shown). 

A  key  detoxification  enzyme  in  astrocytes,  glutamine  synthetase  (GS)  can 
detoxify  the  exitotoxins  glutamate  and  ammonia  and  its  expression  is  modulated  by 
changes  in  the  surroundings  of  astrocytes.  However,  employing  Western  blot  analysis 
we  found  little,  if  any,  changes  of  GS  expression  in  U87  cells  exposed  to  the  conditions 
employed  (data  not  shown). 

Growth  factors  and  neuronal  survival  enhancers  are  putatively  released  into  the 
medium  by  astrocytes  to  protect  their  surrounding  neurons.  To  further  investigate  this 
neuroprotective  mechanism,  we  collected  media  in  which  we  grew  U87  cells  under  the 
conditions  specified  in  Figure  1  above  and  assessed  their  ability  to  influence  survival  of 
SK-N-SH  cells  employing  the  MTT  cell  survival  assay  (Fig.  3).  Our  results  suggested 
that  exposure  of  SK-N-SH  cells  to  various  combinations  of  SF  media  allowed  them  to 
survive  better  than  when  cultured  in  10%  FBS  medium  (control  condition)  alone  (Fig.  3). 
Specifically,  we  found  that  exposure  to  PZ  enhanced  this  protective  effect,  even  under 
conditions  of  oxidative  stress. 

In  conclusion,  results  from  this  study  suggest  changes  in  cell  morphology  of 
astrocytes-like  U87  cells  in  various  environmental  conditions  (e.g.,  serum  deprivation  & 
oxidative  stress)  can  be  correlated  with  up-regulation  of  GFAP  in  U87  cells. 
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The  Effect  of  Different  Ratio  of  Regular  If  edia  and  Conditioned  lledia  on 
SK-N-SH  cells  with  and  without  BSO,  PZ,  and  BSO  +  PZ  in  24  hrs 


Regular/Conditioned  Media  Ratio 


Figure  3.  Treatment  effects  on  SK-N-SH  cell  survival. 

Our  findings  also  suggest  that  conditioned  medium  has  the  ability  to  enhance  the 
survival  of  neurons-like  SK-N-SH  cells  and  PZ  treatment  modulates  this  effect.  Thus, 
our  results  provide  additional  evidence  that  our  cell  model  can  be  productively 
employed  to  further  elucidate  the  putative  neuroprotective  mechanisms  of  astrocytes. 
As  such  they  assume  pathophysiological  importance  in  neuronal  survival/death 
mechanisms  in  neurodegenerative  diseases  such  as  AD  and  PD.  Clearly,  this  is  an  area 
that  merits  further  study. 
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Differential  Cytotoxic  Effects  of  Magnesium  oxide  Nanoparticles  on  Cisplatin 

Sensitive  and  Cisplatin  Resistant  Leukemia  Cancer  Cells 

Abstract: 

Magnesium  oxide  (MgO)  has  many  applications  in  pharmaceutical,  cosmetic  and 
food  industry.  It  has  been  extensively  used  in  these  industries  for  decades.  Magnesium 
oxide  is  used  in  preparation  of  antacid,  laxatives  and  mineral  supplements.  It  is  also 
used  in  cosmetics  in  preparation  of  sunscreens,  toothpastes  and  dental  cements. 

Earlier  studies  from  our  laboratory  have  characterized  the  effects  of  different  metal 
oxide  nanoparticles  on  normal  fibroblast  cells  and  cancer  cells.  In  this  study,  we 
compared  the  cytotoxicity  of  magnesium  oxide  nanoparticles  on  murine  leukemia  cells 
(LI  210)  and  cisplatin  resistant  sub  line  (L1210/DDP).  We  found  that  magnesium  oxide 
nanoparticles  exhibit  differential  cytotoxicity  in  LI 210  and  L1210/DDP  leukemia  cells. 
Our  results  also  suggested  that  cisplatin  resistant  cancer  cells  were  cross  resistant  to 
MgO  nanoparticles.  These  findings  may  assume  pathophysiologic  importance  in 
elucidation  of  mechanisms  underlying  anti-cancer  drug  resistance  and  may  aid  future 
discovery  of  anti-cancer  drugs. 

Key  Words:  cancer,  resistance,  nanoparticles,  leukemia 

Introduction: 

Approaches  to  treat  cancers  include  surgery,  chemotherapy  and  radiation  therapy. 
Conventional  chemotherapeutic  agents  like  cisplatin,  methotrexate,  and  5-flurouracil  are 
used  extensively  in  the  treatment  of  a  variety  of  cancers.  One  major  reason  for  the 
failure  of  chemotherapy  is  resistance  to  primary  chemotherapeutic  agents  used  in 
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treatment  (Gottesman  et  al.  2002).  Certain  cancers,  which  develop  resistance  to 
primary  cancer  chemotherapeutic  agent(s),  also  develop  cross  resistance  to  other 
unrelated  chemotherapeutic  agents  (Bhushan  etal.  1999). 

Several  mechanisms  contribute  to  the  development  of  resistance  in  cancers.  Some 
of  the  mechanisms  include  increased  efflux  of  the  drug  out  of  the  cancer  cell,  decreased 
influx  of  into  the  cancer  cell,  and  increase  in  altered  cellular  targets  (Siddik  et  al.  2003, 
Liu  etal.  2009).  Other  studies  implicate  the  involvement  of  altered  cellular  signaling  in 
the  development  of  resistance  to  chemotherapy  in  cancer  cells. 

Overcoming  cancer  drug  resistance  is  a  major  hindrance  in  effective  use  of 
chemotherapeutic  drugs  to  treat  cancers.  Development  of  resistance  in  cancer  cells  is 
the  major  cause  of  failure  of  chemotherapy  in  treatment  of  cancer  (Giaccone  et  al. 

1996).  Better  understanding  of  the  drug  resistance  mechanisms  is  necessary  so  that 
novel  strategies  can  be  devised  for  more  effective  treatment  of  cancer.  In  this  study,  we 
use  a  novel  approach  to  target  resistance  in  cancer  by  using  metal-based 
nanomaterials. 

Metallic  nanoparticles  are  a  subtype  of  nanoparticles  which  comprise  of 
nanoparticles  made  from  metals  and  their  compounds  (e.g.,  titanium  dioxide,  zinc  oxide, 
MgO,  gold,  silver).  The  use  of  nanomaterials  provides  new  technology  and  approaches 
to  develop  drugs  for  treatment  of  diseases.  The  metallic  nanoparticles  differ  from  their 
bulk  counterparts  in  several  properties  like  magnetic,  surface  electric,  and  their  optical 
properties  owing  to  their  small  size.  These  properties  make  metallic  nanoparticles 
suitable  candidates  for  developing  treatment  therapies  and  diagnostic  and  imaging 
agents  (Bhattacharya  et  al.  2008). 
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Work  in  our  laboratory  emphasizes  the  biological  and  toxicological  effects  of 
nanomaterials.  Previous  studies  from  our  laboratory  have  examined  the  effects  of 
different  metallic  nanoparticles  on  neurotumor  cells  and  normal  human  fibroblasts.  We 
found  while  zinc  oxide  and  titanium  oxide  nanoparticles  induced  high  cytotoxicity  in 
normal  fibroblasts,  magnesium  oxide  nanoparticles  exhibited  comparatively  low 
cytotoxicity  on  these  cells  even  at  high  concentrations  (Lai  et  al.  2008).  Because  MgO 
nanoparticles  exhibited  low  cytotoxicity  on  normal  fibroblasts,  we  selected  them  to 
determine  if  they  could  be  used  to  overcome  drug  resistance  in  cisplatin  resistant 
leukemia  cancer  cells.  If  resistant  cells  are  susceptible  to  nanoparticles  in  influencing 
their  proliferation,  we  can  exploit  this  putative  effect  to  develop  novel  ways  to  overcome 
anti-cancer  drug  resistance. 

Materials  and  Methods:  MgO  nanopowder  (Sigma-Aldrich  Cat  no:  549649,  50  nm 
particle  size  nanopowder)  was  dispersed  in  100  ml  of  sterile  saline  in  a  sealed  conical 
flask  and  stirred  at  ambient  temperature  for  four  hours  prior  to  be  diluted  to  the  specified 
concentrations  for  treatment  of  cells.  McCoy’s  5A  medium  was  purchased  from  Sigma- 
Aldrich.  Mouse  polyclonal  antibodis  for  Akt  and  Erk,  respectively,  were  obtained  from 
Santa  Cruz  Biotechnology. 

Cells  and  Culture  Conditions:  LI 210  murine  leukemia  cells  were  purchased  from 
Sigma-Aldrich  and  maintained  under  sterile  conditions  at  37°C  and  5  %  CO2  in  McCoy’s 
5A  supplemented  with  10%  (v/v)  horse  serum.  The  L1210/DDP  murine  leukemia  cells 
were  obtained  as  a  subline  of  LI 210  cells  resistant  to  cisplatin  and  were  maintained 
under  sterile  conditions  at  37°C  and  5%  CO2  in  McCoy’s  5A  supplemented  with  10% 
(v/v)  fetal  bovine  serum. 


4 


Page  438  of  531 


Dose  Response  Assay:  LI  210  and  L1210/DDP  cells  were  split  at  equal  density  in 
each  well  of  24-well  plates.  Cells  in  each  well  were  treated  with  specified  concentrations 
of  MgO  nanoparticles  (ranging  from  0.1  pg/ml  to  100  pg/ml)  for  72  hours  at  37°C.  After 
72  hours,  an  aliquot  was  used  for  counting  using  a  Z2  Beckman  coulter  counter. 
Western  Blot  Analysis:  LI 210  cells  were  treated  with  various  concentrations  of  MgO 
nanoparticles  (10,  25,  50,  or  100  pg/ml)  for  72  hours.  After  treating  the  cells  with  MgO 
nanoparticles,  cell  lysates  were  prepared  using  the  lysis  buffer  (1%  (v/v)  Triron  X-100, 
10  mM  Tris  base  pH  7.6,  5  mM  EDTA,  50  mM  sodium  chloride,  30  mM  sodium 
pyrophosphate,  50  mM  sofium  fluoride,  0.1%  (w/v)  sodium  azide,  50  mM  phenyl  methyl 
sulphonyl  fluoride,  0.5  mg/ml  aprotinin,  2.5  mg/ml  leupeptin,  and  100  mM  sodium 
orthovanadate  in  distilled  water,  pH  7.6).  BioRad  reagents  (Bradford  assay)  were  used 
to  determine  the  protein  concentration.  25  pg  of  protein  sample  was  loaded  onto  the 
wells  in  the  sodium  dodecyl  sulphate  polyacrylamide  gel.  The  proteins  separated  by  gel 
electrolysis  were  transferred  to  polyvinylidene  fluoride  membrane  (PVDF),  blocked  (5% 
(w/v)  Tris  buffered  saline  with  0.01%  (v/v)  Tween  orTBST  in  no-fat  powdered  milk). 
After  washing,  the  blots  were  incubated  in  1:500  primary  antibody  solution  prepared  in 
5%  bovine  serum  albumin  (BSA)  in  distilled  water.  Membranes  were  washed  and 
treated  with  the  secondary  antibody  and  developed  using  chemiluminescence  kit 
(Pierce  biotechnology,  Rockford,  Illinois),  as  recommended  by  the  manufacturer.  The 
blots  were  analyzed  for  the  levels  of  Akt,  pAkt,  Erk  and  pErk.  The  scans  were  digitized 
suing  Unscan-it-gel  6.1  software 
Results: 
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Dose  Response  Studies:  LI 210  and  L1210/DDP  cells  were  treated  with  different 
concentrations  of  magnesium  oxide  nanoparticles  for  72  hours.  The  dose  response 
studies  were  performed  following  the  procedure  described  above. 


L1210/DDP  treated  with  MgO  nanoparticles 


Concentration  of  MgO  nanoparticles  in  pg/ml 


■  LL210/DDP 
treated  with 
MgO 

ifanopa  rticles 


Figure  1:  Dose  response  assay  of  L1210/DDP  cells  treated  with  MgO  nanoparticles 
after  72  hours. 


The  results  revealed  that  at  the  concentrations  employed,  MgO  nanoparticles  did  not 
exert  any  cytotoxic  effect  on  L1210/DDP  cells  after  72  hours  treatment  (Figure  1).  By 
contrast,  MgO  nanoparticles  did  exert  some  cytotoxic  effects  on  LI 210  cells  after  72 
hours  treatment  (Figure  2).  When  the  effects  of  MgO  nanoparticles  on  the  two  cell  lines 
were  compared  (Figure  3),  it  was  evident  that  the  resistant  L1210/DDP  cells  was  not 
susceptible  to  the  cytotoxicity  of  the  nanoparticles  at  the  concentrations  used  while  the 
sensitive  LI  210  cells  showed  dose-related  decrease  in  survival  upon  treatment  with  the 
nanoparticles. 
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L1210  treated  with  MgO  nanoparticles 
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Figure  2:  Dose  response  assay  of  LI  210  cells  treated  with  MgO  nanoparticles 
after  72  hours. 
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Figure  3:  Dose  response  assays  of  LI  210  and  L1210/DDP  cells  after  72  hours  treatment. 


Akt  and  Erk  expression:  We  performed  western  blot  analysis  to  assess  the  change  in 
expression  of  cell  signaling  elements  like  Akt,  pAkt,  Erk  and  pErk  in  LI  210  cells  after 
MgO  nanoparticles  treatment  using  a  procedure  described  above.  Our  results  showed 
that  the  expression  of  pAkt  and  pErk  in  these  cells  decreases  when  treated  with 
increasing  concentrations  of  MgO  nanoparticles  (Figure  4). 
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Figure  4:  Western  Blot  Analysis  of  the  effect  of  MgO  nanoparticels  on  expression  of  AKT, 
pAKT,  Erk  and  pErk  in  L1210  cells.  In  the  above  figure  1,2,3, 4  and  5  represent  the 
protein  expression  levels  at  control,  10  pg/ml,  25  pg/ml,  50  pg/ml  and  100  pg/ml 
treatment  of  MgO  nanoparticles. 

In  future  we  will  examine  the  effects  of  these  nanoparticles  on  expression  of 
other  cell  signaling  proteins  like  multiple  drug  resistance  associated  protein. 
Discussion:  The  major  reason  for  the  failure  of  cancer  chemotherapy  is  the 
development  of  resistance  to  chemotherapy  in  cancer  cells.  Few  drug  resistance 
mechanisms  for  cisplatin,  which  contribute  to  cisplatin  resistance,  include  increase  in 
intracellular  glutathione,  increase  in  MRP  protein  expression,  and  increase  in  DNA 
repair  mechanisms.  Despite  these  suggested  mechanisms,  a  definitive  mechanism  for 
cisplatin  drug  resistance  remains  to  be  demonstrated.  In  our  laboratory,  we  are 
developing  a  novel  approach  towards  using  metal-based  nanoparticles  in  overcoming 
cancer  drug  resistance. 

Although  magnesium  oxide  nanoparticles  did  not  overcome  drug  resistance  in 
L1210/DDP  cells,  our  results  from  the  dose-response  studies  showed  that  the  cisplatin 
resistant  L1210/DDP  leukemia  cancer  cells  are  cross  resistant  to  MgO  nanoparticles. 
We  also  observe  that  cisplatin  sensitive  leukemia  cells  are  sensitive  to  MgO 
nanoparticles.  These  observations  suggest  that  L1210/DDP  may  have  similar 
mechanisms  of  drug  resistance  for  cisplatin  and  for  MgO  nanoparticles. 
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Altered  expression  of  signal  transduction  pathways  are  known  to  play  a  role  in 
drug  resistance.  Our  analysis  of  cell  signaling  proteins  like  Akt,  pAkt,  Erk  and  pErk  by 
western  blot  analysis  showed  that  the  MgO  nanoparticles  decreased  the  expression  of 
phosphorylated  forms  of  Akt  and  Erk  in  the  sensitive  cell  line.  The  decrease  in  the  levels 
of  phosphorylated  forms  of  Akt  and  Erk  with  increasing  concentrations  of  MgO 
nanoparticles  in  LI 210  cells  is  highly  suggestive  of  down-regulation  of  pathways  leading 
to  cellular  proliferation  and  other  processes  critical  for  cancer  progression.  Clearly,  this 
is  an  important  area  that  merits  further  investigation. 
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Abstract  Use  of  nanoparticles  in  cancer  diagnosis,  detection,  and  therapy  is  rapidly  gaining 
impetus.  This  review  focuses  on  the  recent  advances  in  the  applications  of  metallic  nanoparticles 
in  cancer  imaging  and  therapy.  In  particular,  we  discuss  the  uses  of  iron  oxide  nanoparticles, 
gadolinium-based  nanoparticles,  and  multimodal  metallic  nanoparticles  in  cancer  imaging  and 
detection.  Subsequently,  we  discuss  the  applications  of  several  types  of  cancer  therapies  based 
on  the  use  of  metallic  nanoparticles.  The  latter  applications  include:  metallic  nanoparticles  in 
photodynamic  therapy,  in  hyperthermia  therapy,  and  in  neutron  capture  therapy  as  well  as 
surface-modified  metallic  nanoparticles  in  cancer  therapy.  Despite  the  possibility  that  advances 
in  biomedical  applications  of  such  nanoparticles  may  ultimately  lessen  the  suffering  of  cancer 
patients  and  improve  their  prospects  for  survival,  we  raise  a  strong  note  of  caution  regarding  the 
safety  of  clinical  use  of  metallic  nanoparticles  because  of  their  putative  or  demonstrated  toxicity. 


Key  Words:  Metallic  nanoparticles;  cancer  imaging;  cancer  therapy 

1.  Introduction 

The  diverse  applications  of  nanotechnology  in  biomedical  research  and  development  have 
escalated  exponentially  in  the  last  decade.  This  generalization  is  particularly  pertinent  when  one 
considers  the  use  of  nanomaterials  in  the  development  of  targeted  drug  delivery  systems  [1],  In 
light  of  growing  interests  in  this  and  related  areas,  this  review  aims  to  highlight  and  discuss  the 
recent  and  current  advances  of  applications  of  one  class  of  nanoparticles,  namely  metallic 
nanoparticles,  in  cancer  imaging  and  therapy. 
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2.  Applications  of  Metallic  Nanoparticles  in  Cancer  Imaging 

Although  various  other  techniques  (e.g.,  X-ray  imaging,  CT-Scans,  ultrasound  imaging)  exist, 
this  review  focuses  on  the  current  use  of  metallic  nanoparticles  (including  iron  oxide 
nanoparticles  such  as  nanoparticles  of  super  paramagnetic  iron  oxides  (SPIO)  and  ultrasmall 
superparamagnetic  iron  oxides  (USPIO),  gadolinium  nanoparticles,  and  manganese 
nanoparticles)  in  magnetic  resonance  imaging  (MRI)  in  the  detection  and  staging  of  cancer  [2], 
2.1  Iron  Oxide  Nanoparticles 

Two  subtypes  of  iron  oxide  nanoparticles  are  employed  in  the  MRI  applications  —  colloidal  iron 
oxide  nanoparticles  and  USPIO. 

Colloidal  Iron  Oxide  Nanoparticles 

Colloidal  iron  oxides,  also  called  superparamagnetic  iron  oxide  (SPIO)  nanoparticles  constitute 
the  first  generation  iron  oxide  nanoparticles  and  are  comprised  of  colloidal  iron  oxides.  Prepared 
by  one-step  alkaline  precipitation  of  iron  oxides  (average  size  ~75  nm),  they  are  stabilized  by 
addition  of  polymers  [3,  4], 

Blood  half  life  of  SPIO  nanoparticles  is  short  because  they  are  rapidly  phagocytozed  by 
mononuclear  phagocyte  system  (MPS),  leading  to  their  accumulation  in  normal  tissues  but  not  in 
tumors  [5].  This  differential  localization  of  phagocytozed  SPIO  nanoparticles  facilitates  the 
noninvasive  and  easy  detection  of  tumors  in  the  body  by  MRI. 

SPIO  nanoparticles  have  been  successfully  administered  orally  for  imaging  of  the 
gastrointestinal  tract  [6].  Ferumoxsil  (Lumirem)  (i.e.,  SPIO  nanoparticles  coated  with  silicone)  is 
approved  for  detecting  cancer  metastasis  to  the  liver  [7].  SPIO  nanoparticles-enhanced  MRI  is 
superior  to  helical  CT  to  detect  liver  metastases  in  candidates  for  hepatic  resection  [8]  and 
superior  to  ultrasound  and  CT  in  detecting  focal  hepatic  lesions  and  suspected  cancer  lesions  [9]. 
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Additionally,  SPIO  nanoparticles-enhanced  MRI  scans  proved  to  be  beneficial  in  planning 
surgeries  of  squamous  cell  carcinoma  of  head  and  neck  [10]. 

Ferucorbotran  (Resovist),  another  SPIO  formulation,  is  currently  under  Phase  III  clinical  trials  in 
USA  for  detection  of  liver  neoplasms.  Ferumoxide  (Endorem,  Feridex)  is  an  FDA-approved  MRI 
contrasting  agent  for  detection  of  neoplastic  lesions  in  liver  neoplasms. 

Ultrasmall  Superparamagnetic  Iron  Oxide  (USPIO)  Nanoparticles 
The  second  generation  iron  oxide  nanoparticles,  USPIO  nanoparticles  are  smaller  in  size  (<50 
nm):  consequently,  their  circulating  half  life  is  increased,  thereby  enabling  them  to  permeate 
through  the  vascular  endothelium  and  reach  the  interstitium  and  the  lymph  nodes  [11].  In  the 
lymph  nodes,  USPIO  nanoparticles  are  rapidly  phagocytosed  by  lymphocytes:  their  differential 
accumulation  in  lymph  nodes  gives  rise  to  a  contrast  in  the  imaging  of  normal  versus  metastatic 
lymph  tissues  using  MRI. 

Staging  of  cancer  is  essential  for  planning  its  treatment.  In  contrast  to  the  current  less  precise 
techniques  of  CT  and  ultrasound,  MRI  using  ferumoxtran-10  can  identify  the  spread  of  cancer  to 
lymph  nodes  with  better  sensitivity  and  specificity  [12],  MRI  using  USPIO  nanoparticles  helps 
identify  patients  requiring  salvage  radiation  therapy  after  prostatectomy  [13].  USPIO 
nanoparticles  (i.e.,  ferumoxytol-10)-enhanced  MRI  potentially  can  detect  brain  tumors  [14], 

2.2  Gadolinium 

For  some  time,  gadolinium  chelates  have  been  used  as  a  MRI  contrast  agent  [15].  Recently 
gadolinium-based  nanparticles  have  been  introduced  into  these  MRI  applications.  Gadolinium 
oxide  nanoparticles  with  fluorescence  probes  have  been  synthesized  for  use  as  contrast  agents  for 
MRI  and  fluorescence  imaging  [16].  Silica  nanoparticles  loaded  with  gadolinium  have  been  used 
to  detect  nasopharyngeal  cancer  in  nude  mice  [17].  Gadolinium  dendrimers  are  being  researched 
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for  their  efficiency  as  MRI  contrast  agents:  the  efficiency  of  gadolinium  delivery  depends  upon 
the  type  of  dendrimer  used.  Comparative  study  of  gadolinium  delivery  to  lymph  nodes  using 
polyamidoamine  (PMAM)  and  diaminobutane  (DAB)  dendrimers  showed  that  PMAM 
dendrimers  were  the  more  efficient  in  delivering  gadolinium  to  the  lymph  nodes  [18],  thereby 
demonstrating  their  potential  use  in  detecting  lymph  node  involvement  of  tumors  by  MRI 
imaging  [19].  However,  the  drawback  associated  with  the  use  of  PMAM  dendrimers  labeled  with 
gadolinium  is  the  intrinsic  toxicity  associated  with  PMAM  and  gadolinium  [20,  21], 

2.3  Multimodal  Metallic  Nanoparticles 

Multi-modal  imaging  is  sensitive,  specific  and  yields  more  information  than  single-probe 
imaging.  Multimodal  metallic  nanoparticles  consist  of  magnetic  nanoparticles  as  the  core  to 
which  different  functional  moieties  are  tagged.  The  functional  moieties  can  be  fluorescence 
probes  for  fluorescence  image  analysis,  antibodies  for  targeting  a  particular  group  of  cells,  or 
biomarker  and  radionucleotides  for  PET  imaging  [22],  For  example,  polysialic  acid  (PSA)  is  a 
cancer  biomarker  for  neuroblastoma.  HmenBl  antibody  tagged  with  magnetic  iron  oxide 
nanoparticles  having  a  dye  doped  silica  core  (for  fluorescence  imaging)  can  target  PSA,  thereby 
facilitating  the  detection  of  neuroblastoma  in  rats  [23].  Similarly,  iron  oxide  nanoparticles  tagged 
with  a  fluorescence  probe  have  been  investigated  for  breast  cancer  detection  in  nude  mice  [24], 

It  is  important  to  note  that  the  current  research  trend  on  applications  of  metallic  nanoparticles  in 
cancer  imaging  strongly  suggests  that  such  applications  can  be  coupled  to  or  employed  in 
conjunction  with  applications  of  metallic  nanoparticles  in  cancer  therapy. 

3.  Application  of  Metallic  Nanoparticles  in  Cancer  Therapy 

Conventional  cancer  therapies  include  chemotherapy,  radiation  therapy,  and  surgery.  These 

therapies  have  limitations  that  decrease  their  efficacy:  anti-cancer  drugs  exhibit  adverse  effects 
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on  normal  cells  and  cross-resistance  to  other  anti-cancer  drugs  [25]  and  radiation  therapy  also 
show  deleterious  side  effects  [26].  Thus,  improved  cancer  therapies  are  needed.  Metallic 
nanoparticles  have  demonstrated  potential  in  several  experimental  cancer  therapies,  including 
photodynamic  therapy  (PDT),  hyperthermia  therapy  (HT),  neutron  capture  therapy  (NCT),  and 
surface-modified  metallic  nanoparticles  for  drug  delivery  or  anti-proliferative  activity  [27]. 

3.1  Metallic  Nanoparticles  in  Photodynamic  Therapy 

Photodynamic  therapy  (PDT)  generally  exploits  a  photosensitive  compound  that  is  subsequently 
activated  using  an  external  light  source  to  kill  the  cancer  cells  [28].  When  photons  from  the  light 
source  are  incident  upon  the  photosensitive  material,  the  photosensitive  drug  is  excited  and 
produces  reactive  oxygen  species  that  damage  the  cancer  cells.  Choice  of  the  photosensitive 
compound  depends  upon  the  location  of  the  tumor  in  the  body  [29,30]. 

Gold  nanoparticles  conjugated  with  epidermal  growth  factor  receptor  (EGFR)  antibodies  can 
induce  cancer  cell  death  after  irradiation  with  light  in  the  near-infrared  (NIR)  range.  The 
conjugation  of  anti-EGFR  antibodies  to  spherical  gold  nanoparticles  facilitates  their  selective 
accumulation  in  cancer  cells  that  over-express  EGFR  receptors  [30].  The  temperatures  achieved 
upon  NIR  irradiation  of  gold  nanoparticles  in  vitro  were  70°  -  80°  C  [31]. 

A  comparison  of  the  photothermal  destruction  of  malignant  cancer  cells  using  gold  nanoparticles 
with  different  surface  dimensions  and  core  material  revealed  that  the  efficiency  of  gold 
nanoparticles  was  dependent  on  the  photothermal  conversion  efficiency  and  dimensions  of 
nanoparticles  [32],  Moreover,  gold  nanoshells  can  be  used  effectively  for  photodynamic  therapy 
in  cancer  [33]  and  Au-Ag  nanorods  functionalized  with  aptamer  exhibit  specificity  of  these 
nanorods  and  their  use  for  PDT  in  treating  cancer  [34], 
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The  advantages  of  PDT  include:  1)  it  is  a  noninvasive  technique  for  killing  cancer  cells;  2)  it 
induces  tumor  regression  prior  to  surgery;  and  3)  it  can  target  tumors  located  at  inoperable  areas 
(e.g.,  trachea).  The  main  disadvantages  of  PDT  therapy  include,  but  are  not  limited  to:  1)  it  uses 
near-IR  light  of  limited  power;  2)  it  does  not  penetrate  more  than  1  cm  deep  into  the  tissue;  and 
3)  PDT  therapy  induces  reddening  or  inflammation  of  the  target  tissue. 

3.2  Metallic  nanoparticles  in  Hyperthermia  Therapy 

Hyperthermia  therapy  (HT)  is  effected  by  exposing  the  cancerous  tissues  to  high  temperatures  of 
up  to  42-43°C  leading  to  damage  or  killing  of  cancer  cells.  Unlike  PDT,  where  a  light  source  is 
used  to  induce  rise  in  target  temperatures,  HT  utilizes  radiofrequency,  ultrasound,  or  microwave 
for  raising  the  target  temperatures  [35].  HT  is  a  conjugate  therapy  employed  when  the  tumors  are 
nonresponsive  to  chemotherapy  and  radiation  therapy. 

Conventional  HT  employing  heating  sources  like  microwave,  radiofrequency  or  ultrasound  is 
non-specific  [36].  Metallic  nanoparticles  can  help  overcome  this  drawback:  thus,  the  use  of 
magnetic  nanoparticles  in  HT  was  proposed  [37].  Therapeutic  potentials  of  iron  oxide 
nanoparticulate  formulations  are  being  actively  assessed  in  HT  [38]. 

Use  of  magnetic  iron  oxide  nanoparticles  for  HT  is  called  -Magnetic  Fluid  Hyperthermia” 
(MFH)  [39].  When  iron  oxide  nanoparticles  (dimensions  <10  nm)  are  subjected  to  an  alternating 
external  magnetic  field,  the  heat  so  generated  ultimately  damages  cancer  cells.  An  in  vitro  and  in 
vivo  study  demonstrated  this  MFH  effect  of  Fe2C>3  nanoparticles  in  cultured  liver  cancer  cells  and 
in  xenograph  liver  cancers  [40].  Similarly,  MFH  effect  of  dextran  coated  iron  oxide  and  that  of 
silane-coated  nanoparticles  were  studied  in  normal  and  cancerous  human  cells  in  vitro :  the 
consequent  hyperthermia  induced  cytotoxicity  in  cancer  cells  [41].  Furthermore,  conjugating 
iron  oxide  nanoparticles  with  folate,  which  binds  to  its  receptors  expressed  by  cancer  cells,  is 
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effective  in  inducing  hyperthermia  within  the  cancer  cells  [42],  MFH  effect  of  Fe203,  in 
combination  with  arsenic  trioxide,  was  noted  in  human  cervical  cancer  cells  [43]  and  Au 
nanoparticles  produced  marked  hyperthermia  in  gastrointestinal  cancer  cells  in  vitro  [44], 

Despite  their  utility  in  inducing  hyperthermia,  a  major  drawback  of  this  application  is  that  the 
temperature  generated  thus  inside  cells  is  difficult  to  control.  A  new  class  of  magnetic 
nanoparticles,  manganite  perovskites  show  promise  to  overcome  this  limitation.  The  curie 
temperatures  (42°  -  43°C)  of  silver-doped  manganite  pervoskites  enable  isothermal  heating  [45]. 
3.3  Metallic  Nanoparticles  in  Neutron  Capture  Therapy 

Neutron  capture  therapy  utilizes  a  stable  nuclide  delivered  to  the  tumor  cells.  Upon  irradiation 
with  neutrons,  the  stable  nucleus  produces  ionizing  radiations  [46].  Boron  Neutron  capture 
therapy  (BNCT)  is  a  form  of  radiotherapy  which  utilizes  neutron  beam  and  boron-10:  irradiation 
with  neutron  rays  induces  boron  to  form  lithium  and  alpha  rays.  Although  BNCT  using  boron- 10 
was  investigated  for  treating  glioblastoma  multiforme  [47],  progress  of  BCNT  in  cancer 
treatment  has  been  slow  because  few  tumor-selective  boron-containing  drugs  exist  and  the 
neutron  beams  employed  are  of  the  low-energy  type  [48,  49]. 

Use  of  gadolinium  for  NCT  was  recognized  over  two  decades  ago.  Gadolinium  emits  gamma 
rays  when  irradiated  with  neutrons  [50].  Currently,  gadolinium  nanoparticles  (Gd-NP)  are  being 
invesgated  for  their  application  in  NCT  [51].  A  comparison  of  Gd-NP  and  gadolinium  solution 
revealed  that  Gd-NP  showed  significant  accumulation  and  tumor  growth  suppression  in  cancer 
cells  as  compared  to  gadolinium  solution  [52],  Gd-NP  can  also  target  cancer  cells  by  attaching  to 
Gd-NP  ligands  or  antibodies  specific  to  the  cancer  cells.  By  attaching  folate  to  the  surface  of  Gd- 
NP,  the  uptake  of  Gd-NP  into  cancer  cells  in  vitro  and  in  vivo  is  enhanced  [46,  53]. 
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3.4  Surface-modified  metallic  nanoparticles  in  cancer  therapy 

Another  type  of  application  of  metallic  nanoparticles  in  cancer  therapy  exploits  the  intrinsic 
properties  of  the  nanoparticles.  That  silver  has  been  extensively  commercialized  in  applications 
as  anti-microbial  agents  [54]  prompted  investigations  of  anti-proliferative  effect  of  silver 
nanoparticles  on  human  glioblastoma  cells  [55].  Though  the  underlying  mechanisms  are 
incompletely  understood,  some  researchers  believe  this  anti-proliferative  effect  is  attributable  to 
oxidative  stress  induced  by  Ag  nanoparticles  [56].  Radioactive  gold  nanoparticles  coated  with 
gum  arabica  glycoprotein  show  promising  results  in  targeting  and  treating  prostate  cancer:  the 
nanoparticles  passed  through  tumor  vasculature  to  induce  decrease  in  tumor  volume  [57]. 

One  adverse  effect  of  radiation  therapy  is  its  deleterious  effects  on  normal  cells.  Cerium  oxide 
nanoparticles  have  demonstrated  selective  radio-protective  effect  on  normal  cells  [58].  For 
example,  UV  exposure  of  normal  and  tumor  breast  cells  pretreated  with  cerium  oxide 
nanoparticles  induced  more  deaths  in  cancer  cells  compared  to  normal  cells.  Although  the  radio 
protective  effect  of  nanoceria  in  normal  cells  is  not  understood,  some  researchers  hypothesize 
this  effect  is  due  to  superoxide  dismutase  enzyme  mimetic  activity  of  nanoceria  [59]. 

Gadolium  endohederal  metallofullerenol  nanoparticles  have  exhibited  high  anti-tumor  activity  in 
mice  bearing  liver  tumors  [60].  Gadolinium  endohederal  metallofullerenol  nanoparticles  owe 
their  anti -tumor  activity  to  their  ability  to  scavenger  reactive  oxygen  species  [61,  62],  Metallic 
nanoparticles  of  motexafin  gadolinium  also  have  similar  antioxidant  properties.  Clinical  studies 
have  indicated  that  they  can  prevent  metastasis  of  cancer  to  brain  when  administered  to  patients 
in  early  stages  of  non-small  cell  lung  cancer  (NSCLC)  [63,  64], 

Metallic  nanoparticles  have  improved  the  delivery  of  chemotherapeutic  drugs,  specifically  at  the 
tumor  site.  Metallic  nanoparticles  like  iron  oxide  nanoparticles  which  have  magnetic  properties 
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can  be  targeted  to  the  tumor  site  under  the  influence  of  external  magnetic  field  [65].  For 
example,  employing  iron  oxide  nanoparticles  as  carriers  for  chemotherapeutic  drugs  such  as 
doxorubicin  and  camptothecan  resulted  in  improved  therapeutic  efficacy  of  these  drugs  in 
decreasing  tumor  size  in  rat  cancer  xenograft  model  [66]. 

4.  Conclusions  and  Future  Directions 

The  current  rapid  and  explosive  advances  of  nanotechnology  herald  a  new  age  in  cancer 
diagnosis  and  therapy.  In  this  review,  we  have  only  highlighted  the  seminal  roles  of  metallic 
nanoparticles  in  cancer  imaging  and  therapy.  Clearly  one  should  appreciate  the  advances  we 
have  discussed  above  constitute  only  a  small  fraction  of  the  applications  of  nanotechnology  in 
cancer  imaging  and  therapy.  However,  limitation  of  space  does  not  allow  us  to  even  mention 
other  important  advances  unrelated  to  the  use  of  metallic  nanoparticles. 

Although  mortality  rates  due  to  cancer  have  shown  decreases  in  the  recent  two  decades,  cancer  is 
still  the  leading  cause  of  disease-related  deaths  in  USA.  The  advances  in  development  of  newer 
cancer  imaging  and  therapies  based  on  metallic  nanoparticles  may  help  in  early  detection  of 
cancer  and  thus  contribute  to  decreasing  deaths  due  to  cancer.  Various  cancer  imaging  and 
therapies  based  on  use  of  metallic  nanoparticles  are  at  different  stages  of  preclinical  and  clinical 
development.  Iron  oxide  nanoparticles,  zinc  oxide  nanoparticles,  gold  nanoparticles,  silver 
nanoparticles,  and  cerium  oxide  nanoparticles  have  tremendous  potentials  to  be  developed  as 
novel  diagnostic  and  therapeutic  agents  in  cancer. 

Enhanced  cancer  biomarker  and  genetic  mutation  detection  techniques  would  help  in  identifying 
individuals  at  high  risk  for  developing  cancer.  In  this  context,  multi-functional  metallic 
nanoparticles  show  exciting  therapeutic  potentials  and  these  are  currently  under  development  for 
cancer  therapy  to  be  clinically  applied  in  the  near  future.  Metallic  nanoparticles  can  be 
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engineered  to  enhance  the  efficacy  of  current  diagnostic  and  imaging  techniques  in  cancer. 
Furthermore,  cancer  therapies  utilizing  metallic  nanoparticles  as  targeting  vehicles  for  drug 
delivery,  and  other  allied  therapies  using  metallic  nanoparticles  would  also  play  key  roles  in 
clinical  management  of  cancer  in  the  next  decade. 

Understandably,  the  advances  of  the  biomedical  applications  of  metallic  nanoparticles  discussed 
above  strongly  suggest  that  the  clinical  applications  of  such  nanoparticles  may  ultimately  lessen 
the  suffering  of  cancer  patients  and  improve  their  prospects  for  survival.  Nevertheless,  we  cannot 
help  but  inject  a  strong  note  of  caution  here.  As  the  ubiquitous  and  diverse  applications  of 
nanotechnology  have  become  almost  uncontrollable  phenomena,  humans  are  increasing  exposed 
to  nanoparticles  and  other  nanomaterials  although  the  health  effects  and  environmental  impact  of 
their  exposure  have  not  been  systematically  assessed  and  are  therefore  largely  unknown  [67]. 
Moreover,  recent  advances  in  understanding  the  putative  toxicity  of  nanoparticles  and  other 
nanomaterials  [67-72]  indicate  that  in  developing  metallic  nanoparticles  for  cancer  imaging  and 
therapy,  one  needs  to  consider,  and  fully  investigate  in  parallel,  the  putative  toxicity  of  such 
nanoparticles  prior  to  even  adopting  them  for  pre-clinical  trial. 
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ABSTRACT 

Cell  culture  models  in  vitro  have  long  served  as  tools  for 
the  elucidation  of  cellular  and  molecular  mechanisms  of 
diseases.  Recently  these  versatile  models  have  gained  wide 
acceptance  in  toxicology  and  tissue  engineering  research. 
Although  these  models  are  not  exactly  the  same  as  the 
models  in  vivo ,  they  facilitate  mechanistic  insights 
pertaining  to  a  particular  cell  type.  Nevertheless,  co¬ 
cultures  of  two  different  but  functionally  complementary 
cell  types  provide  structural  and  functional  perspectives 
single-cell-type  models  do  not.  We  have  initiated  a 
systematic  development  of  this  model  type  employing 
dorsal  root  ganglion  (DRG)  neurons  and  Schwann  cells. 
Our  first  model  involves  non-contact  co-cultures  of 
Schwann  cells  and  DRG  neurons  employing  the  hanging 
cell  culture  insert.  The  results  from  our  studies  to  date 
demonstrate  that  our  non-contact  co-culture  model  is 
suitable  for  both  high  throughput  and  mechanistic  studies  in 
nanotoxicological  and  pathophysiological  research. 

Keywords.  Co-culture  model,  DRG  neurons,  Schwann  cells, 
nanotoxicology,  tissue  engineering,  pathophysiology. 


Topic : 

1  INTRODUCTION 

The  aim  of  our  studies  is  to  systematically  develop  and 
characterize  cell  culture  models  of  neural  cells  of  the 
peripheral  nervous  system  in  vitro  for  nanotoxicological 
and  pathophysiological  research.  Once  such  models  are 
developed,  they  may  have  wide  applications  in  tissue 
engineering  research.  The  need  for  such  models  is  derived 
from  the  fact  that  the  peripheral  nervous  system  is  most 
susceptible  for  the  development  of  neuropathy.  Some 
examples  of  peripheral  neuropathies  include  drug- 
(chemo therapy)  induced  neuropathy  [1],  disease-induced 
neuropathy  (e.g.,  diabetes)  [2],  immune  and  inflammatory 
neuropathies,  inherited  and  genetic  neuropathies,  and  other 
neuropathies  induced  by  toxicity. 

The  applications  of  nanomaterials,  including 
nanoparticles,  have  been  increasing  exponentially  in  the  last 
decade  and  have  infiltrated  diverse  fields.  Once  they  enter 
the  body  by  inhalation  or  other  routes,  nanoparticles  may  be 
toxic  to  one  or  more  organ  systems  [3].  However,  the 
toxicity  of  nanoparticles  in  mammals,  especially  humans, 
has  not  been  systematically  studied  although  there  is 
increasing  concern  about  the  environmental  and  health 
impact  of  exposure  to  nanoparticles  of  different  types  [4]. 
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Consequently,  we  have  systematically  initiated  a  series  of 
studies  to  elucidate  the  putative  cytotoxicity  of 
nanoparticles  in  various  mammalian  cell  types  [4].  For 
example,  once  recent  study  of  ours  noted  that  several  metal 
oxide  (including  titanium  and  zinc  oxides)  nanoparticles 
exert  differential  cytotoxic  effects  on  human  neural  cells  in 
cell  model  systems  in  vitro  [4]  . 

There  have  been,  however,  extremely  few  cell  model 
systems  developed  employing  peripheral  neural  cell  types 
for  nanotoxicological  and  tissue  engineering  studies.  We 
have  therefore  initiated  a  systematic  development  of  this 
type  of  models  employing  Schwann  cells  and  dorsal  root 
ganglion  (DRG)  neurons.  The  first  model  we  have 
developed  involved  a  non-contact  co-culture  model  of 
Schwann  cells  and  DRG  neurons  using  the  hanging  cell 
culture  insert.  Thus,  this  type  of  co-culture  cell  systems, 
consisting  of  two  distinctly  different  but  functionally 
complementary  cell  types,  provide  structural  and  functional 
perspectives  that  single-cell-type  models  do  not. 

We  hypothesized  that  non-contact  co -culture  model  of 
Schwann  cells  and  DRG  neurons  can  be  useful  in 
elucidating  the  putative  toxic  effects  of  various 
nanoparticles  on  the  peripheral  nervous  system. 
Furthermore,  the  putative  cytotoxic  effects  of  nanoparticles 
could  also  influence/disrupt  the  cell-to-cell  communications 
between  the  two  cell  types.  To  test  our  hypothesis,  we  have 
initially  characterized  the  Schwann  cells  the  DRG  neurons 
in  monotypic  or  single-cell-type  cultures.  In  our  ongoing 
studies,  employing  our  new  co-culture,  non-contact  model, 
we  have  recently  demonstrated  that  the  survival  of  DRG 
neurons  is  increased  when  co-cultured  with  Schwann  cells 
in  this  construct.  In  this  study,  we  compare  the  putative 
cytotoxic  effects  of  Si02  and  Ti02  nanoparticles  on  dorsal 
root  ganglion  (DRG)  neurons  and  Schwann  cells  employing 
established  cytotoxicity  testing  [5-8].  Furthermore,  we  are 
investigating  the  putative  toxic  effect  of  these  nanoparticles 
on  interaction(s)  between  DRG  neurons  and  Schwann  cells 
in  non-contact  co-culture. 


2  MATERIALS  AND  METHODS 

2.1  Cells  and  Culture  Conditions 

The  immortalized  DRG  neurons  were  a  gift  from 
Dr.  Ahmet  Hoke’s  Laboratory  at  Johns  Hopkins  University 
(Baltimore,  MD,  USA)  [9].  The  rat  Schwann  cell  line  was 
obtained  from  ATCC  (Manassas,  VA,  USA).  These  cells 
and  their  co-culture  were  cultured  in  DMEM  (Sigma;  St 
Louis,  MO,  USA)  supplemented  with  10%  (v/v)  fetal 
bovine  serum  (FBS),  1%  (w/v)  sodium  pyruvate,  0.292  g/1 
L-glutamine,  1.5  g/1  sodium  bicarbonate,  1%  (v/v)  anti¬ 
mycotic  and  various  specified  concentrations  (5-45  mM)  of 
glucose. 

2.2  Co-culture  Model  System 


DRG  neurons  were  cultured  on  a  Corning  Costar 
24- well  plate.  Schwann  cells  were  then  seeded  on  a 
Millipore  hanging  cell  culture  insert  (pore  size  0.4  pm)  on  a 
separate  plate  and  then  introduced  to  a  DRG  neuron- 
containing  well  after  4  hours  to  achieve  the  co -culture 
condition  [7].  DRG  neurons  (co-cultured  with  Schwann 
cells)  were  exposed  to  various  concentrations  of  Si02 
nanoparticles  for  specified  periods  and  their  survival 
determined  employing  the  cell  survival  assay. 


2.3  Cell  Survival  (MTT)  Assay 

DRG  neurons  or  Schwann  cells  were  seeded  per 
well  in  a  4  8 -well  plate  in  DMEM  with  specified 
concentrations  of  Si02  nanoparticles.  After  incubation  at 
37°C  for  24  or  48  hours,  50  pi  of  MTT  (5  mg/ml  in  PBS) 
reagent  was  added  to  each  well.  After  incubation  for 
another  4  hours,  the  medium  was  removed  gently  and  the 
cellular  reaction  product  was  solubilized  in  200  pi  DMSO. 
Then  the  optical  density  of  the  contents  of  each  well  was 
measured  in  a  plate  reader  at  570  nm  [4].  The  absorbance 
corresponds  to  live  cells  present  in  each  well  [4]. 


112  i 


DRG  neurons  cultured  DRG  neurons  co- 
alone  cultured  with 

Schwann  cells 

figure  1:  Co-cultures  of  DRGTieurons  and  Schwann  cell? 
using  hanging  cell  culture  inserts  (non-contact  co-culture). 
Values  are  mean  ±  SEM  of  2-3  determinations.  DRG 
neurons  cells  treated  with  Si02  nanoparticles  for  24  hours 
are  arked  with  blue  line  and  those  treated  for  48  hrs  are 
marked  with  red  line. 

2.4  Cellular  morphology 

DRG  neurons,  Schwann  cells  and  their  co-culture 
was  treated  with  Si02  specified  concentration  for  24  and  48 
hours  at  37°  C  as  described  above  and  bright  field  images 
were  acquired  by  using  a  Leica  light  microscope  (Leica  DM 
IRB,  Bannockburn,  IL,  USA)  equipped  with  a  digital 
camera  (Leica  DFC  300FX)  [4]. 
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2.5  Statistical  analysis  of  results 

Statistical  significance  of  experimental  results  was 
analyzed  with  one-way  ANOVA  followed  by  Dunnett’s 
post-hoc  test  with  a  minimum  significance  level  set  at  p< 
0.05  using  the  SPSS  17  software  package. 


3  RESULTS  AND  DISCUSSION 

We  are  the  first  to  employ  dorsal  root  ganglion  (DRG) 
neurons  and  Schwann  cells  to  develop  co-culture  models  in 
vitro  because  DRG  neurons  are  the  most  susceptible  nerve 
cells  to  developing  pathologies  in  peripheral  neuropathy. 
We  have  characterized  the  two  cell  types  individually  with 
respect  to  their  optimum  glucose  requirement  and  activities 
of  enzymes  important  in  modulation  of  glutamate- 
glutamine  cycling  (e.g.,  glutamine  synthetase)  that  is  part  of 
the  metabolic  and  functional  inter-dependence  between 
DRG  neurons  and  Schwann  cells  known  to  occur  in  vivo 
(data  not  shown).  We  have  also  studied  the  protein 
expression  of  various  apoptosis-related  proteins  such  as 
BCL-2,  BCL-XL,  BAX  and  cytohrome  C  in  the  two  cell 
types  (data  not  shown). 


Figure  2:  Effects  of  Si02  nanoparticles  on  DRG  neurons 
and  Schwann  cells.  Values  are  mean  ±  SEM  of  12 
determinations  and  replicate  experiments  show  the  same 
trend.  DRG  neurons  and  Schwann  cells  treated  with  Si02 
nanoparticles  for  24  hours  are  marked  red  and  blue, 
respectively,  treated  with  Si02  nanoparticles  for  24  hrs  are 
marked  with  blue  line. 

To  our  knowledge,  we  are  the  first  to  determine,  in  this 
study,  the  putative  cytotoxic  effects  of  Si02  and  Ti02 
nanoparticles  on  neural  cells  of  the  peripheral  nervous 
system  (i.e.,  DRG  neurons  and  Schwann  cells). 


As  shown  in  Figure  2,  treatment  of  DRG  neurons  or 
Schwann  cells  for  24  hours  with  Si02  induced  a  dose- 
related  decreases  in  survival  of  those  cells  when  we 
progressively  increased  the  concentrations  of  the 
nanopartcles  from  1  to  200  pM,  with  IC50  values  of  -150 
pM  (Figure  2).  At  the  highest  concentration  employed,  less 
than  40%  of  the  cells  survived  (Figure  2). 

Because  we  had  previously  noted  that  Ti02 
nanoparticles  exerted  differential  cytotoxic  effects  on 
central  nervous  system  neural  cells  [4],  we  have 
investigated  the  possibility  that  these  nanoparticles  may 
also  exert  cytotoxic  effects  on  neural  cells  from  the 
peripheral  nervous  system.  As  shown  in  Figure  3,  our  on¬ 
going  studies  indicate  that  treatment  with  Ti02 
nanoparticles  in  the  dose  range  so  far  employed  (0.1  to  20 
pM)  appeared  to  lower  the  survival  of  both  DRG  neurons 
and  Schwann  cells:  the  effects  seemed  somewhat  dose- 
related.  Moreover,  the  Schwann  cells  appeared  to  be  more 
susceptible  to  the  effect  of  Ti02  nanoparticles,  suggesting 
they  may  exert  differential  effects  on  the  neural  cells  of  the 
peripheral  nervous  system. 

We  have  also  employed  light  microscopy  to  examine 
the  effects  of  nanoparticles  on  DRG  neurons  and  Schwann 
cells  and  assess  the  effects  of  Si02  and  Ti02  nanoparticles 
thereon  (data  not  shown).  In  general,  the  photomicrographs 
revealed  that  treatment  with  the  nanparticles  induced  dose- 
related  changes  in  cellular  morphology  consistent  with  the 
nanoparticle -induced  progressive  decrease  in  cell  viability 
(Figures  2  and  3). 

As  we  have  previous  noted,  treatment  with  Ti02 
nanoparticles  induced  dose-related  increases  in  cell  death  of 
human  central  neural  cells,  marked  by  apoptosis,  and 
necrosis,  and  an  as  yet  uncharacterized  cell  death 
mechanism  that  appeared  to  be  a  mixture  of  apoptosis  and 
necrosis  [4].  Thus,  the  results  of  our  ongoing  studies  with 
the  peripheral  nervous  system  neural  cells  such  as  DRG 
neurons  and  Schwann  cells  appear  to  show  some  similarity 
to  those  we  obtained  with  central  neural  cells  [4].  The  DRG 
neurons  and  Schwann  cells  appear  also  to  be  susceptible  to 
the  cytotoxic  effects  of  Si02  and  Ti02  nanoparticles.  As  we 
have  indicated  earlier,  we  are  also  determining  the  cell 
death  mechanisms  underlying  the  cytotoxic  effects  of  Si02 
and  Ti02  nanoparticles  on  DRG  neurons  and  Schwann 
cells.  Clearly,  this  is  an  important  area  that  merits  further 
study. 


4  CONCLUSITIONS 

Thus,  the  results  from  our  studies  to  date  as  well  as 
those  from  our  ongoing  studies  demonstrate  that  our  non- 
contact  co-culture  model  is  highly  suitable  for  both  high 
throughput  and  mechanistic  studies  in  nanotoxicological 
and  tissue  engineering  research.  Moreover,  this  model  is 
also  relevant  for  the  investigation  of  mechanistic  issues 
associated  with  peripheral  diabetic  neuropathy  and  other 
disease  states  of  the  peripheral  nervous  system. 
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Figure  3:  Effects  of  Ti02  nanoparticles  on  DRG  neurons  or 
Schwann  cells.  Values  are  mean  ±  SEM  of  12 
determinations  and  replicate  experiments  show  the  same 
trend.  DRG  neurons  and  Schwann  cells  treated  with  Ti02 
nanoparticles  for  24  hours  are  marked  red  and  blue, 
respectively. 


Results  of  our  present  and  ongoing  studies  have 
confirmed  that  neural  cells  of  the  peripheral  nervous 
system,  similar  to  their  counter  parts  in  the  central  nervous 
system,  are  also  susceptible  to  the  cytotoxicity  of  Si02  and 
Ti02  nanoparticles.  Thus,  these  results  are  consistent  with 
our  hypothesis  that  our  non-contact  co-culture  model  of 
Schwann  cells  and  DRG  neurons  can  be  useful  to  study 
toxic  effects  of  various  nanoparticles  on  peripheral  nervous 
system. 

The  results  of  this  and  our  ongoing  studies  suggest  that 
nanoparticles  may  exert  differential  cytotoxic  effects  on 
neural  cell  types  of  the  peripheral  nervous  system.  As  such, 
our  results  may  assume  pathophysiological  importance  in 
the  environmental  health  impact  of  nanoparticles. 
Obviously,  the  is  an  important  area  that  deserves  further 
study. 
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ABSTRACT 

Evidence  is  accumulating  that  cell  culture  models  in 
vitro  facilitate  high  throughput  and  mechanistic  studies  in 
tissue  engineering.  Astrocytes  are  known  to  protect  neurons 
against  pathophysiological  assaults  in  vivo  and  in  vitro 
although  the  underlying  mechanisms  are  not  fully 
understood.  Previously,  we  investigate  several  of  the 
putative  neuroprotective  properties  of  astrocytes  in  cell 
culture  model.  Those  studies  led  us  to  hypothesize  that 
different  stress  factors  can  elicit  and/or  enhance  the 
neuroprotective  effects  of  astrocytes  through  the  activation 
of  astrocytic  signaling  and  alteration  of  astroglial  function. 
The  results  of  this  study  suggest  that  U87  (astrocytes-like) 
astrocytoma  cells  exert  some  protective  effects  on  SK-N-SH 
(neurons-like)  neuroblastoma  cells  against  pathophysiolo¬ 
gical  assaults  (e.g.,  oxidative  stress)  under  several  sets  of 
culture  conditions  not  previously  studied.  Thus,  our  results 
may  have  pathophysiological  implications  in  neuroprotection 


and  implications  and  applications  in  tissue  engineering  and 
nanotoxicological  research. 

Key  Words:  Cell  culture  models;  human  neural  cells;  U87 
astrocytes-like  astrocytoma  cells;  neuroprotection;  tissue 
engineering;  nanotoxicology 

Topic  Area: 

1.  INTRODUCTION 

There  is  increasing  evidence  that  cell  model  systems  in 
vitro  play  vital  roles  in  recent  tissue  engineering  research. 
The  additional  structural  and  functional  perspectives  of  co¬ 
culture  cell  systems  in  vitro  start  to  gain  prominence  over 
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single  cell  type  cell  systems  because  unlike  the  latter,  the 
former  more  closely  resemble  cell  types  in  a  tissue/organ  in 
vivo. 

Astrocytes  (non-nerve,  glial  cells)  in  the  brain  play 
important  roles  in  protecting  neurons  (nerve  cells)  against 
various  assaults;  yet,  the  cellular  and  molecular  mechanisms 
underlying  this  protection  is  far  more  being  understood  [1]. 
Among  the  research  on  further  elucidating  such  mechanisms, 
more  efforts  appear  to  center  on  commonly  known 
neurodegenerative  diseases  such  as  Alzheimer’s  disease 
(AD),  followed  by  Parkinson’s  disease  (PD)  [2].  AD  is  a 
progressive  and  irreversible  neurodegenerative  disorder  in 
human  brain  leading  to  key  symptoms  of  memory  loss  [3]. 
No  known  cure  is  available  for  AD  as  current  therapy 
primarily  treats  AD  symptoms  and  presents  them  from 
worsening.  Thus,  there  is  an  urgent  need  to  better  understand 
how  astrocytes  exert  their  protective  effects  on  neurons  in  a 
co-culture  cell  system  in  vitro.  Once  this  protection  is 
elucidated,  then  one  could  design  treatment  strategies  and/or 
tissue  engineering  application  to  prevent  the  occurrence  of 
accelerated  neuronal  loss  in  neurodegenerative  diseases  such 
as  AD  and  PD. 

Our  project  seeks  to  examine  the  underlying 
neuroprotective  properties  offered  by  astrocytes  to  protect 
neurons,  employing  a  co-culture  cell  model  system  in  vitro. 
We  have  been  developing  several  model  systems  to 
investigate  putative  cytotoxic  and  inflammatory  mechanisms 
associated  with  induction  of  neuronal  cell  damage  and  cell 
death  [4-6].  Our  previously  data  have  shown  that  U87 
astrocytoma  and  SK-N-SH  neuroblastoma  cells  constitute 
good  co-culture  model  systems  for  astrocytes  and  neurons, 
respectively,  in  vitro  because  of  their  close  functional 
resemblance  to  normal  astrocytes  and  neurons  and  that  U87 
cells  protect  SK-N-SH  cells  from  several  pathological 
assaults  [4-5].  We  therefore  hypothesized  that  different 
stress  factors  can  elicit  and/or  enhance  the  neuroprotective 
effects  of  astrocytes  through  the  activation  of  astrocytic 
signaling  and  alteration  of  astroglial  function.  In  this  study, 
we  have  investigated  this  hypothesis  further. 

2.  MATERIALS  AND  METHODS 

2.1  Cell  culture  of  U87  and  SK-N-SH  cells 

The  human  astrocytoma  (U87)  and  neuroblastoma 
(SK-N-SH)  cell  lines  were  obtained  from  (ATCC;  Manassas, 
VA,  USA)  and  cultured  in  MEM  supplemented  with  10% 
(v/v)  fetal  bovine  serum  (Atlanta  Biologicals;  Lawrence ville, 
GA),  1%  (w/v)  sodium  pyruvate  (Sigma  Aldrich;  St  Louis, 
MO),  0.292  g/L  L-glutamine  (Sigma),  1.5g/L  sodium 
bicarbonate  (Sigma)  and  1%  (v/v)  antimycotic  (Atlanta 
Biologicals;  Lawrenceville,  GA).  Cells  were  maintained  at 
37°C  and  5%  (v/v)  C02  as  described  previously  [4-5]. 

2.2  Treatment  of  U87  cells 


The  U87  cells  were  cultured  in  T-75  tissue  culture 
flasks  to  -50%  confluent  in  20  ml  of  10%  LBS -containing 
MEM  medium.  Then  the  medium  was  discarded  and 
changed  to  20  ml  of  5%  PBS -containing  medium  and  the 
cells  incubated  for  24  hours.  After  that,  the  medium  was 
replaced  with  20  ml  of  medium  containing  0%  PBS  and  the 
cells  incubated  for  an  additional  24  hours.  Cells  were  treated 
L-sulfoximine  buthionine  (BSO),  or  pioglitazone  (PZ)  or  a 
combination  of  both.  At  the  end  of  another  24  hours  of 
incubation,  the  media  were  removed  and  frozen  with  liquid 
nitrogen  immediately.  The  cell  pellet  was  collected  and  both 
cell  pellet  and  media  collected  were  kept  at  -80°C  until  used 
for  experiments. 

2.3  Cell  Survival  (MTT)  assay 

The  neuroprotective  properties  of  U87  cells  on  SK- 
N-SH  cells  were  determined  in  different  combinations  of 
regular  medium  (RM)  and  conditioned  medium  (CM) 
collected  from  U87  cells  cultured  under  various  conditions 
(see  above).  RC/CM  ratio  of  100/0,  70/30,  50/50,  40/60, 
30/70,  and  0/100  were  used.  SK-N-SH  cell  survival  after  the 
treatments  was  assessed  employing  the  MTT  assay  [5]. 

2.4  Western  Blot  Analysis 

Western  blot  analysis  was  employed  to  assess 
protein  expression  in  cell  lysates  to  determine  the  effects  of 
media,  BSO  and  PZ.  SDS-PAGE  electrophoresis  was 
performed  as  previously  described  [7].  The  separated 
proteins  were  visualized  using  chemiluminesence  detection 
kit  while  p-actin  was  the  loading  control. 


3.  RESULTS  AND  DISCUSSION 

Our  culture  model  consists  of  U87  (astrocytes-like) 
cells  and  medium  collected  from  U87  cells  cultured  under 
specified  conditions  to  grow  SK-N-SH  (neurons-like)  cells. 
This  is  a  new  modified  co-culture  strategy  to  allow  the 
characterization  of  putative  “protective  factors”  secreted  by 
U87  cells  to  protect  SK-N-SH  cells  in  a  non-contact  cell 
culture  setup.  In  addition,  we  can  characterize  the  putative 
structural  and  functional  properties  of  U87  cells  under  the 
different  specified  conditions. 

A  selective  inhibitor  of  glutathione  (naturally- 
occurring  antioxidant)  synthesis,  L-buthionine  sulfoximine 
(BSO),  is  used  to  induce  oxidative  stress  in  cells.  A 
peroxisome  proliferator  activated  receptor  gamma  (PPARy) 
agonist,  Pioglitazone  (PZ),  is  used  to  block  the  inflammation 
which  is  usually  unregulated  in  AD  [8,9].  We  have  designed 
studies  whereby  we  could  utilize  BSO  and  PZ  in  addition  to 
serum  depletion  to  further  elucidate  the  putative  protective 
mechanism(s)  of  U87  cells  on  neurons-like  SK-N-SH  cells. 
Serum  depletion  and  drug  treatment  for  U87  cell  cultures 
were  prepared  as  shown  in  the  flow  chart  below  (Fig.  1). 
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U87  cells  cultured  in  medium  containing  10%  FBS  were 
used  as  control  (not  shown  in  Fig.  1).  Medium  containing 
0%  FBS  were  defined  as  serum  free  (SF)  medium. 


U-87  cells  in 
1 0%  FBS 
Medium 


U-87  cells  in 
5%  FBS 
Medium 


|  -  |  | 


U-87  cells  in  SF 
Medium 

1 

U-87  cells  in  SF 
Medium 

+  BSO 

1 

U-87  cells  in  SF 
Medium 

-*-PZ 

1 

U-87  cells  in  SF 
Medium 

+  BSO  +PZ 

Figure  1.  Experimental  design  of  U87  cells  treated  with  BSO, 
PZ  or  in  combination.  The  U87  cells  were  grown  to  -50% 
confluent  in  10%-FBS  containing  medium  and  subsequently 
discarded  and  changed  to  5%  FBS -containing  medium  and 
incubated  for  24  hours.  After  that,  the  medium  was  replaced 
with  medium  containing  0%  FBS  and  incubated  for  an 
additional  24  hours.  Cells  were  treated  BSO  or  PZ  or  a 
combination  of  both  for  another  24  hours  of  incubation. 

Morphological  and  functional  changes/adaption  in 
astrocytes  in  vivo  and  in  vitro  is  known  to  associate  with 
oxidative  and  other  environmental  stresses.  We  employed 
light  microscopy  to  examine  the  effects  of  BSO  treatment 
and  FBS  depletion  to  induce  oxidative  and  environmental 
stress,  respectively,  on  U87  cells  and  assess  the  effects  of  PZ 
treatment  thereon. 


employed  western  blot  to  determine  if  the  morphological 
changes  in  U87  cells  exposed  to  SF  medium  with  and 
without  BSO/PZ  treatment  were  seen  also  associated  with 
changes  in  GFAP  expression.  An  increased  GFAP 
expression  was  observed  when  U87  cells  were  cultured  in  SF 
medium  compared  to  those  cultured  in  10%  FBS  medium 
(data  not  shown),  suggesting  a  correlation  between  changes 
in  cellular  morphology  and  GFAP  expression. 

Growth  factors  and  neuronal  survival  enhancers  are 
putatively  released  into  the  medium  by  astrocytes  to  protect 
their  surrounding  neurons.  To  investigate  this  possibility, 
we  collected  media  in  which  we  grew  U87  cells  under  the 
conditions  specified  in  Figure  1  above  and  assessed  their 
ability  to  influence  survival  of  SK-N-SH  cells.  Our  results 
suggested  that  exposure  of  SK-N-SH  cells  to  various 
combinations  of  SF  media  allowed  them  to  survive  better 
than  when  cultured  in  10%  FBS  medium  (control  condition) 
alone  (Fig.  3).  Moreover,  exposure  to  PZ  enhanced  this 
protective  effect,  even  under  conditions  of  oxidative  stress. 


4.  CONCLUSIONS 

When  taken  together,  our  results  suggest  astrocytes¬ 
like  U87  cells  changed  their  cell  shape  and  morphology 
when  cultured  under  serum- free  media  with  and  without 
drug  treatment.  Those  morphological  changes  correlated 
with  upregulation  of  GFAP  in  such  cells. 
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Figure.  2  Treatment-induced 
cellular  morphology  changes 
in  U87  cells  (100X 
magnification).  Top  panel  of 
photomicrographs  represents 
U87  cells  cultured  in  10% 
FBS  medium  treated  with  or 
without  drug  or  drug 
combination  while  those  of 
the  bottom  panel  represents 
U87  cells  cultured  in  serum- 
deprived  (SF)  medium 
treated  with  or  without  drug 
or  drug  combination. 


Treatment  with  SF  medium  induced  U87  cells  to 
put  out  multipolar  processes  and  the  other  treatments 
accentuated  this  effect  (Fig.  2).  Clearly,  these  observations 
raise  interesting  mechanistic  questions  that  need  to  be 
addressed  further  (see  below). 

Glial  fibrillary  acid  protein  (GFAP)  is  a  well-known 
and  classical  marker  for  astrocytes:  its  altered  expression  is 
associated  with  changes  in  astroglial  morphology.  Thus,  we 


Moreover,  exposure  of  SK-N-SH  cells  to  various 
combinations  of  SF  media  allowed  them  to  survive  better 
than  when  cultured  in  10%  FBS  medium  (control  condition) 
alone  and  exposure  to  PZ  enhanced  this  protective  effect, 
even  under  conditions  of  oxidative  stress.  Thus,  our  results 
provide  additional  evidence  that  our  cell  model  can  be 
productively  employed  to  further  elucidate  the  putative 
neuroprotective  mechanisms  of  astrocytes.  As  such  they 
assume  pathophysiological  importance  in  neuronal 
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survival/death  mechanisms  in  AD  and  PD.  Clearly,  this  is  an 
area  that  merits  further  study,  especially  in  tissue 
engineering  and  nanotoxicological  research  and  applications. 
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Ligure  3.  Treatment  effects  on  SK-N-SH  cell  survival  with  different  regular/conditioned  media  combination  (see  Tig.  1).  SK-N- 
SH  cells  cultured  in  10%  LBS  served  as  the  control. 

Values  are  expressed  as  %  of  the  control  and  are  given  as  mean  ±  SEM  of  separate  determinations.  *p<0.05  versus  control. 
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ABSTRACT 

Chitosan  exhibits  antimicrobial  activities  through  its 
interaction(s)  with  microbial  cell  surface  thereby  altering  their 
gene  expression  and  cellular  function,  leading  to  cell  death. 
Nanometal  particles  exert  many  effects  not  previously 
expected  on  biological  systems  and  thus  can  be  explored  for 
diverse  biomedical  applications.  Our  previous  and  on-going 
studies  indicate  that  U87  cells  cultured  on  chitosan 
film/membrane  exhibited  significantly  slower  growth  and 
proliferation  kinetics  compared  to  U87  cells  cultured  alone.  In 
this  study  we  tested  the  hypothesis  that  the  inhibitory  effect  of 
chitosan  is  enhanced  if  combined  with  nanometals  and 
adriamycin,  a  common  anticancer  durg.  Our  results  showed 
that  combinations  of  metal  nanoparticles,  adriamycin  and 
chitosan  induced  decreased  survival  of  U87  glioma  cells  at 
different  rates,  more  marked  than  those  with  chitosan  alone. 
Thus,  our  results  have  pathophysiological  implications  in 
inhibiting  human  brain  glioma  invasion  and  migration. 

Key  words:  Chitosan,  nanoparticles,  adriamycin,  cancer 
therapy,  U87  cells 

1  INTRODUCTION 

Chitosan,  a  polysaccharide  biopolymer  that  combines  a 
unique  set  of  physicochemical  and  biological  properties  has 
increasingly  gained  popularity  in  biomedical  applications  [1, 
2].  Our  previous  and  on-going  studies  indicate  that  U87  cells 
(human  brain  glioblastoma  cell  line)  cultured  on  chitosan 
film/membrane  exhibite  significantly  slower  growth  and 
proliferation  kinetics  compared  to  U87  cells  cultured  in  the 
absence  of  chitosan  film/membrane  [3].  Chitosan  exhibits 
antimicrobial  activities  through  its  interaction(s)  with 
microbial  cell  surface  thereby  altering  their  gene  expression 
and  cellular  function  and  leading  to  cell  death  [4]. 


Nanometal  particles,  such  as  nanosilver,  nanogold,  are 
important  material  being  utilized  frequently.  Recent  studies  of 
nanometal  particles  have  revealed  many  properties  that  were 
not  previously  expected  in  biological  systems  and  thus  can  be 
explored  for  various  applications  in  biomedicine,  such  as 
tissue  engineering  [5]  and  wound  dressing  [6]. 

In  this  study,  we  hypothesized  that  the  inhibitory  effect  of 
chitosan  would  be  greatly  modulated  if  we  combine  chitosan 
with  nanometals  and  adriamycin,  a  common  drug  for  cancer 
therapy.  Similar  treatments  with  a  different  cancer  (PANC-1, 
human  pancreatic  cancer  cell  line)  and  normal  cells  (BJ, 
human  fibroblast  cell  line)  were  also  used  to  investigate  this 
hypothesis. 

2  MATERIALS  AND  METHODS 

2.1  Materials 

Human  astrocytoma  (astrocytes-like)  U87  cells,  human 
pancreatic  PANC-1  cells  and  human  fibroblast  BJ  cells  were 
obtained  from  ATCC  (Manassas, VA,  USA).  Chitosan  (from 
crab  shells,  minimum  85%  deacetylated),  thiazolyl  blue 
tetrazolium  bromide  (MTT),  dimethyl  sulfoxide  (DMSO)  and 
adriamycin  were  purchased  from  Sigma-Aldrich  (St  Louis, 
MO,  USA).  Fetal  bovine  serum  (FBS)  was  obtained  from 
Atlanta  Biologicals  (Lawrenceville,  GA,  USA), 
tetrachloroauric  (III)  acid  (HAuC14*3H20),  trisodium  citrate 
(C6H5Na307*2H20)  and  silver  nitrate  (AgN03)  were 
purchased  from  Fisher  Scientific  (Pittsburgh,  PA,  USA). 
Other  chemicals  were  of  analytical  grade  and  were  usually 
from  Sigma-Aldrich  (St.  Louis,  MO,  USA)  unless  otherwise 
stated. 

2.2  The  Preparation  of  Nanosilver,  Nanogold 
Particles 
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To  prepare  nanosilver,  AgN03  and  C6H5Na307*2H20 
solutions  were  filtered  through  a  0.22  pm  microporous 
membrane  filter,  and  nanosilver  was  prepared  according  to 
the  literature  [7]  by  adding  C6H5Na307*2H20  solution  to 
boiling  AgN03  aqueous  solution. 

To  prepare  nanogold,  HAuC14*3H20  and 
C6H5Na307*2H20  solutions  also  need  to  be  filtered  through  a 
0.22  pm  microporous  membrane  filter  prior  to  use.  Nanogold 
was  prepared  according  to  the  literature  [8]  by  adding 
C6H5Na307*2H20  solution  to  boiling  HAuC14»3H20  aqueous 
solution. 

2.3  Cell  Culture 

Chitosan  membrane  was  prepared  as  described  previously 
[3].  A  certain  amount  of  nanosilver  or  nanogold  solution  was 
added  into  each  well  of  24-well  culture  plates  in  which  a 
sterile  chitosan  membrane  had  already  been  placed.  After  12 
hours,  nanosilver  or  nanogold  solution  was  aspirated  and 
sterile  phosphate -buffered  saline  (PBS)  was  added  into  each 
well  to  wash  the  membrane. 

U87  cells  were  seeded  with  equal  density  in  each  well  of 
the  24- well  plates  and  cultured  in  an  incubator  at  37°  C  and  5 
%  C02  in  modified  Eagle’s  medium  (MEM)  supplemented 
with  10%  fetal  bovine  serum  (FBS).  Adriamycin  with  a  final 
concentration  of  0.1  pm  was  added  after  cells  were  seeded. 
Culture  of  PANC-1  cells  and  BJ  cells  were  similar  with  U87 
cells  whereas  PANC-1  cells  were  grown  in  the  RPMI  1640 
medium. 

2.4  MTT  Assay 

Cell  survival  and  growth  was  determined  using  the  MTT 
assay  [9].  Cells  (U87,  PANC-1,  and  BJ)  were  cultured  in  24- 
well  plates  as  described  in  the  preceding  subsection.  At  the 
end  of  the  incubation  period  (4,  7,  10,  or  14  days),  MTT  dye 
(0.5%,  w/v,  in  PBS)  was  added  to  each  well  and  the  plates 
were  incubated  for  an  additional  4  hours  at  37°C.  The  purple - 
colored  insoluble  formazan  crystals  in  viable  cells  were 
dissolved  using  dimethyl  sulfoxide,  and  the  subsequent 


absorbance  of  the  content  of  each  well  was  measured  at  570 
nm  using  a  Bio-Tek  Synergy  HT  Plate  Reader  (Winooski, 
VT,  USA)  [10]. 

2.5  Statistical  Analysis  of  Data 

Results  are  presented  as  mean  ±  standard  error  of  the 
mean  (S.E.M.)  of  6  determinations  in  each  experiment. 

3  RESULTS  AND  DISCUSSION 

As  shown  in  Figure  1,  all  the  treatments  induced 
decrease  in  cell  survival  of  U87  cells.  There  were  apparent 
differences  in  the  effects  exerted  by  different  treatments. 
After  being  treated  for  14  days,  chitosan  with  naosilver  and 
0.1  pm  adriamycin  was  the  most  effective.  This  may  enable 
lower  doses  of  adriamycin  to  be  used,  with  reduced  toxicity. 
Chitosan  with  nanogold  and  0.1pm  adriamycin  was  the 
second  most  effective;  chitosan  alone  was  the  least  effective. 
Chitosan  with  nanosilver  showed  greater  effect  than  chitosan 
with  nanogold.  Therefore,  these  results  provided  some 
support  for  our  hypothesis  that  a  combination  of  chitosan  with 
nanometals  and  adriamycin  was  more  effective  than  chitosan 
or  adriamycin  alone  or  chitosan  with  nanometals. 

As  shown  in  Figure  2,  all  the  treatments  showed  an 
inhibitory  effect  on  cell  survival  of  PANC-1  cells.  Similarly, 
after  being  treated  for  14  days,  chitosan  with  nanosilver  and 
0.1  pm  adriamycin  was  the  most  effective;  chitosan  alone  was 
the  least  effective.  However,  the  effect  was  stronger  on 
PANC-1  cells  than  on  U87  cells.  Compared  with  U87  cells, 
PANC-1  cells  were  considerably  more  sensitive  to  the  effects 
of  adriamycin  alone. 

Figure  3  showed  the  effect  of  different  treatments  on 
the  survival  of  BJ  cells.  Chitosan  with  nanosilver  and  0.1  pm 
adriamycin  was  the  most  effective,  the  same  as  U87  cells  and 
PANC-1  cells.  By  day  14,  however,  there  was  no  significant 
difference  in  the  survival  rate  of  cells  treated  with  chitosan 
alone  and  chitosan  with  nanogold  and  untreated  (control). 

Figure  1:  Effect  of  chitosan,  chitosan/nanosilver, 
chitosan/nanogold  and  adriamycin  on  survival  of 
U87  cells.  Red  circles:  chitosan;  Black  cross 
signs:  chitosan  with  nanogold;  Blue  squares:  0.1 
pm  adriamycin;  Green  squares:  chitosan  with  0.1 
pm  adriamycin;  Red  plus  signs:  chitosan  with 
nanosilver;  Blue  triangles:  chitosan  with 
nanogold  and  adriamycin;  Dark  green  circles: 
chitosan  with  nanosilver  and  0.1  pm  adriamycin. 

At  the  end  of  the  specified  incubation  time,  cell 
survival  and  growth  was  determined  using  the 
MTT  assay.  Values  are  the  mean  ±  SEM. 
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Figure  2:  Effect  of  chitosan,  chitosan/nanosilver, 
chitosan/nanogold  and  adriamycin  on  survival  of 
PANC-1  cells.  Red  circles:  chitosan;  Black  cross 
signs:  chitosan  with  nanogold;  Blue  triangles: 
chitosan  with  nanogold  and  adriamycin;  Green 
squares:  chitosan  with  0.1  jam  adriamycin;  Blue 
squares:  0.1  jam  adriamycin;  Red  plus  signs: 
chitosan  with  nanosilver;  Dark  green  circles: 
chitosan  with  nanosilver  and  0.1  jam  adriamycin. 
At  the  end  of  the  specified  incubation  time,  cell 
survival  and  growth  was  determined  using  the 
MTT  assay.  Values  are  the  mean  ±  SEM. 
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Figure  3:  Effect  of  chitosan,  chitosan/nanosilver, 
chitosan/nanogold  and  adriamycin  on  survival  of 
BJ  cells.  Red  circles:  chitosan;  Black  cross  signs: 
chitosan  with  nanogold;  Blue  squares:  0.1  pm 
adriamycin;  Blue  triangles:  chitosan  with 
nano  gold  and  adriamycin;  Green  squares: 
chitosan  with  0.1  pm  adriamycin;  Red  plus 
signs:  chitosan  with  nanosilver;  Dark  green 
circles:  chitosan  with  nanosilver  and  0.1  pm 
adriamycin.  At  the  end  of  the  specified 
incubation  time,  cell  survival  and  growth  was 
determined  using  the  MTT  assay.  Values  are  the 
mean  ±  SEM. 


Our  previous  and  ongoing  studies 
demonstrate  that  combinations  of  metal 
nanoparticles,  adriamycin  and  chitosan  induced 
cell  death  at  different  rates,  with  reference  to 
U87  cells  cultured  in  the  absence  of  chitosan  and 
with  chitosan  alone.  Chitosan  with  nanosilver 
and  0.1  pm  adriamycin  showed  the  greatest  cell 
survival  reduction  on  all  three  cell  lines. 
Chitosan  with  nanosilver  showed  greater  effect 
than  chitosan  with  nanogold.  Taken  together, 
these  results  suggest  potentials  for 
pathophysiological  applications  in  inhibition  of 
human  brain  glioma  migration  and  invasion  and 
in  treatment  of  pancreatic  cancer. 
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ABSTRACT 

In  our  previous  reports,  we  studied  the  development  of 
biosensor  platform  that  are  capable  of  measuring 
biometabolites  and  environmental  sensitive  species,  such  as 
peroxide  and  nitrate/nitrate,  to  concentration  in  the  order  of 
ppb  (parts  per  billion)  or  lower. 

In  our  more  recent  development,  we  modified  our  platform 
with  dendrimers  to  enhance  its  performance.  Zero  and 
second  generation  of  dendrimers  were  coated  on  the  surface 
of  a  carbon  glassy  platform  electrode  modified  with  GDH 
(glutamate  dehydrogenase)  and  it  was  used  to  measured 
ammonium,  a  common  biometabolite,  at  near  neutral  pH 
that  is  common  for  normal  bioactivities. 

The  resulting  electrode  was  tested  with  ammonium 
concentrations  ranged  from  0.002  to  0.3  jliM  with 
satisfactory  results.  Measurements  at  lower  concentrations 
had  better  resolution  than  at  higher  concentrations  and  it  is 
believed  that  the  lower  concentration  limit  can  be  better 
than  the  tested  lower  limit  at  0.002  pM.  Performance  of  the 
modified  carbon  glassy  electrode  was  compared  with  other 
glassy  electrodes  that  were  modified  differently,  and  the 
results  will  be  reported. 

The  biosensor  platform  thus  far  was  proven  to  be  versatile 
and  can  be  used  in  many  biomedical  and  environmental 
applications. 


Key  words:  biosensor,  nanoparticle,  GDH, 
ammonium,  electrode,  PAMAM 

1  INTRODUCTION 


In  the  past  several  years,  our  research  group  has  been 
developing  a  platform  for  biosensor  construction  that  are 
capable  of  detecting  target  species  that  are  in  nano  and 
subnanomolar  levels  [1,2]. 

In  this  study,  we  were  developing  a  biosensor  that  can 
measure  ammonia  at  submicromolar  levels  that  cannot  be 
detected  by  commercial  ammonia  electrodes  or  wet 
chemistry  method  easily. 


1.1  Theory 

As  shown  in  the  following  equation,  glutamate  and 
NAD+  can  be  hydrolyzed  to  form  a-keto  glutarate,  NADH, 
and  ammonium  ion  with  the  enzyme,  glutamate 
dehydrogenase.  The  equilibrium  constant  is  in  favor  of  the 
formation  of  glutamate  and  thus  the  reverse  reaction  is 
faster  kinetically. 


HC-NH3 

U  +  NAD+ 
^  (or  NADP+) 

C-Cf 

O 

Glutamate 


Glutamate 
d  ehy  dr  og  erase 

+  h2o  ^  nh4+  + 


c-o 

I 

c=o 

ft- 

C-O" 


NADH 
(or  NAD  PH) 


a-Ketoglutarate 


+  H+ 


For  many  biological  reactions,  the  end  product  or 
metabolite  is  ammonium,  accumulation  of  high 
concentration  of  ammonium  is  toxic  to  the  body. 
Environmentally,  ammonium  is  a  byproduct  of  many 
industrial  processes,  uncontrolled  discharges  of  ammonium 
will  lead  to  harmful  consequence  to  the  environment.  Our 
goal  here  was  to  develop  a  sensor  device  that  can  measure 
ammonium  ion  at  the  lowest  concentrations  possible,  its 
utilities  can  be  in  biomedical  or  environmental  applications. 
Hence,  it  is  the  reverse  of  the  reaction  as  shown  above 
(glutamate  formation)  that  we  utilized. 
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2  MATERIALS  AND  METHODS 

2.1  Materials 

L-glutamic  dehydrogenase  (from  bovine  liver,  solution 
in  50%  glycerol)  (GDH)  was  purchased  from  Sigma- 
Aldrich  (St.  Louis,  MO,  USA),  the  concentration  of  GDH  is 
28  mg  protein/ml,  and  46  units/mg.  Cysteamine, 
polyamidoamine  dendrimer  generation  0  (PAMAM  O), 
polyamidoamine  dendrimer  generation  2nd  (PAMAM_2nd), 
and  a-keto  glutarate,  AuC13HC1-4H20  (Au  %>  48%)  and 
Na3citrate  were  purchased  from  Sigma.  All  the  other 
chemicals  were  of  analytical  grade  or  highest  grade 
available.  All  the  experiments  were  carried  out  under 
deoxygenated  condition  in  0.1  M  phosphate  buffer  solution. 

2.2  Electrode  Preparation 

The  cleaned  glassy  carbon  electrode  was  first  immersed 
in  0.1  M  cysteamine  solution  in  darkness.  The  resulting 
monolayer-modified  electrode  was  rinsed  thoroughly  with 
twice-distilled  water  and  soaked  in  water.  Then,  it  was 
dipped  into  the  colloidal  gold.  The  gold  colloid- 
cysteamine-modified  electrode  was  dipped  into  the  1- 
glutamate  dehydrogenase  (GDH)  solution  (pH  7.4)  (or 
GDH  solution  containing  NADH).  In  such  a  way,  a  GDH 
(/NADH)  gold  colloid-cysteamine-modified  glassy  carbon 
electrode  was  obtained. 

In  case  that  both  cysteamine  and  PAMAM  were  coated 
onto  electrodes,  the  cleaned  glassy  carbon  electrode  was 
first  immersed  in  cysteamine  solution,  the  resulting 
electrode  was  dipped  into  the  PAMAM  solution.  The  gold 
colloid-cysteamine/PAMAM-modified  electrode  was 
dipped  into  the  GDH  solution  (pH  7.4)  (or  GDH  solution 
containing  NADH).  In  such  a  way,  a  GDH  (/NADH)  gold 
colloid-cysteamine/PAMAM-modified  glassy  carbon 
electrode  was  obtained. 

2.3  Nanoparticles  Solution  Preparations 

Nanoparticles  Au  was  prepared  by  reacting  HAuC14 
with  citric  acid  [2]. 

2.4  Detections 

UV-VIS  spectrophotometry  was  carried  out  by  an 
Agilent  diodearray  spectrophotometer;  cyclic  voltammetry 
was  conducted  by  using  a  Gamry  600  Potentiostat. 
Voltammetric  potential  was  measured  against  a  saturated 
chloride  electrode  (SCE). 

3  RESULTS  AND  DISCUSSION 


3.1  Stability  of  Electrodes 


Figure  1.  Cysteamine-PAMAM-Au-GDH  coated  on  plastic 
UV-VIS  cell  with  permanent  absorption  peak  at  278  nm. 
Various  lines  indicated  concentration  differences. 

As  shown  in  Figure  1,  PAM  AMs  can  be  attached  to 
cysteamine  to  enhance  the  available  sites  for  the  electrons 
transfer  between  reactions,  in  this  case,  ammonium  and  a- 
keto  glutarate,  therefore  increase  the  detection  lower  limit. 


Figure  2.  Cyclic  voltammograms  (i/v)  of  gold  colloid- 
cysteamine/PAMAM_2ND-modified  glassy  carbon  electrode 
measured  from-0.2  to  0.4  V  for  the  measurement  of  NH4+, 
the  lowest  concentration  was  2  nM. 

Figure  2  shows  the  cyclic  voltammograms  of  various 
ammonium  concentrations  measured  by  the  modified  glassy 
carbon  electrode,  the  lowest  concentration  was  2  nM.  As 
indicated  in  the  voltammogram  in  the  reductive  curves 
(upper  lines),  there  was  a  big  gap  (current  difference) 
between  the  blank  (of  buffer  solution)  and  the  first 
reductive  curve  (2  nM).  Therefore,  it  is  possible  that  the 
modified  electrode  can  measure  ammonium  concentrations 
down  to  subnanomolar  levels. 


real  c one  oNtet-dglutarate/iiM 

Figure  3.  Performance  differences  of  the  glassy  carbon 
electrode  modified  by  different  materials:  1 .  Modified  with 
cysteamine-Au-GDH/NADH;  2.  Cysteamine  /PAMAM0- 
Au-GDH/NADH;  3.  PAMAM_0-Au-GDH/NADH;  4. 
Cysteamine  /PAMAM_2nd-Au-GDH/NADH;  5. 
PAMAM_2nd  -Au-GDH/NADH. 

Figure  3  shows  that  there  were  distinctive  current 
differences  for  the  reaction  of  ammonium  and  a-keto 
glutarate  when  the  electrode  was  modified  by  different 
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materials.  From  the  stand  point  of  detection,  within  the  five 
modifications,  sensor  coated  with  second  generation  of 
PAM  AM  would  have  the  highest  sensitivity  (current  vs. 
concentration).  The  results  were  conceivably  due  to  the 
higher  branching  property  of  PAMAM_2nd. 

4  CONCLUSIONS 

We  successfully  modified  our  glassy  carbon  electrode 
biosensor  platform  [2]  for  ammonium  detection;  we  further 
modified  the  sensor  platform  with  PAMAMs  and  the 
detection  lower  limit  was  enhanced.  This  highly  modified 
electrode  can  detect  ammonium  concentrations  down  to  2 
nM,  or  lower.  The  low  detection  limit  of  this  biosensor  is 
far  more  superior  to  most  available  methods,  more  research 
is  needed  for  noise  reduction  and  the  stability  of  the 
electrode. 
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Abstract.  Lifelong  exposure  to  environmental  factors  can  influence  the  risk  of  developing  diseases  according  to  recent  research 
findings.  Environmental  stresses  ultimately  leading  to  neuronal  cell  death  have  been  hypothesized  as  the  causes  of  the  increased 
occurrence  in  developing  Alzheimef's  and  Parkinson"s  disease.  Our  daily  diet  is  considered  to  be  one  of  the  most  important 
environmental  factors  that  can  seriously  affect  the  development  and  proper  functions  of  the  brain.  Depending  on  the 
concentrations,  metals  and  electrolytes  can  post  some  health  concerns,  especially  for  a  prolonged  consumption  period.  For 
example,  it  was  reported  that  excess  amounts  of  iron,  zinc  and  copper  in  the  human  brain  may  cause  oxidative  damage  and 
protein  aggregation;  the  neurotoxicity  induced  by  these  metals  may  lead  to  cerebral  and/or  cerebellar  degeneration.  Other  reports 
showed  that  there  were  differences  in  concentrations  of  five  different  elements  (aluminum,  zinc,  copper,  manganese,  and  iron) 
between  normal  human  brain  and  brains  of  patients  with  Alzheimer"s  disease.  In  this  study,  we  investigated  30  elements, 
including  electrolytes,  and  how  dietary  intake  on  a  life-time  basis  would  affect  their  concentrations  and  distributions  in  various 
regions  of  the  rat  brain  (hypothalamus,  cerebellum,  pons  and  medulla,  striatum,  mid-brain,  cerebral  cortex,  and  hippocampus) 
and  discussed  their  health  implications.  Information  matrices  of  these  30  different  elements  (mostly  metals)  and  their 
distributions  in  various  regions  of  the  rat  brain  were  analyzed  as  a  function  of  normal  dietary  intake  at  different  ages  during 
development.  Our  results  showed  that  metallomic  distribution  in  various  regions  of  the  rat  brain  is  age-related.  The  results  may 
help  researchers  to  identify  possible  links  between  daily  dietary  intake  of  metals  and  electrolytes  and  diseases  associated  with 
aging  (e.g.,  Alzheimer"s  and  Parkinson"s disease)  and  suggest  such  metallomic  distributions  may  be  used  as  neurological 
biomarkers  of  exposure  to  heavy  metals. 

Keywords :  Brain;  elements;  metals;  electrolytes;  distribution;  metallomic;  neurological;  diseases 


1.  Introduction 

Many  of  the  metals  around  us  are  essential  for  life,  but  others  are  known  to  be  highly  toxic;  even  the  essential 
metals  can  be  toxic  when  their  intakes  are  too  high.  For  example,  metals  such  as  chromium,  cobalt,  copper, 
manganese,  and  zinc  are  essential  for  life.  But,  an  excess  intake  of  these  essential  metals  can  induce  toxicity.  Some 
metals  such  as  arsenic,  cadmium,  lead,  mercury,  and  vanadium,  which  are  found  throughout  our  environment,  are 
toxic  to  humans  and  other  animals.  Some  of  these  toxic  metals  are  even  capable  of  forming  covalent  bonds  with 
carbon,  resulting  in  metal-organic  transformations.  This  type  of  transformation  affects  the  mobility  and  toxicity  of 
the  element.  Elements  can  come  from  naturally-occurring  processes  such  as  volcanoes,  water,  bacterial  activity,  and 
also  from  anthropogenic  sources  such  as  automobile  exhaust,  agricultural  fertilizers,  industrial  activities,  and  many 
other  sources.  It  has  been  shown  that  in  highly  industrialized  areas,  the  exposure  to  metallic  elements  is  extremely 
high  [1]. 

Being  a  specialized  organ  of  the  body,  the  brain  metabolizes  and  accumulates  metals  as  part  of  its  normal 
development  and  function.  But  in  a  rich  metal  environment,  loss  of  metalloproteins  and  loss  of  defense  against 
oxidative  stress  caused  by  one  or  more  of  the  heavy  metals  could  be  responsible  for  neurodegenerative  disorders 
such  as  Parkinson"s  disease  (PD)  and  Alzheimer"s  disease  (AD)  [2].  The  intake  of  these  metals  occurs  via  ingestion 
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of  metal-containing  food  and  water,  and/or  through  inhaling  metal-contaminated  air.  The  elemental  distributions  in 
the  different  brain  regions  appear  to  vary  for  each  element.  Some  of  these  metallic  elements  are  known  to  increase 
or  decrease  in  brains  of  humans  with  a  neurodegenerative  disorder. 

The  following  are  some  of  the  more  studied  elements  in  brain;  other  elements  are  also  found  in  brain,  but 
research  on  the  latter  group  has  been  scant.  Although  many  metals  are  normally  found  in  brain,  over-accumulation 
of  metals  may  lead  to  health  problems.  Moreover,  an  unbalanced  increase  or  decrease  could  cause  a  major  functional 
change  and  lead  to  a  neurodegenerative  disorder  even  though  the  underlying  molecular  mechanisms  are  largely 
unknown. 

Aluminum 

Aluminum  (Al)  is  one  of  the  more  widely  distributed  metals  in  the  environment.  Approximately  8%  of  the  earth"s 
crust  is  Al.  The  exposure  of  Al  normally  occurs  through  air,  food,  and  water  [3].  The  brain  contains  approximately 
1%  of  the  body"s  total  Al;  however,  Al  has  no  known  function  in  normal  mammalian  brain. 

During  the  life-span  of  a  normal  human  brain,  its  Al  levels  appear  to  increase  around  age  40,  and  then  plateau  at 
about  age  70.  The  Al  levels  then  begin  to  increase  again  from  age  80  to  90.  The  globus  pallidus  (GP),  substantia 
nigra  (SN)  and  the  nucleus  ruber  appear  to  be  the  highest  Al  levels  in  the  normal  human  brain  [4].  The  levels  of  Al 
increase  in  the  grey  matter  in  patients  with  dialysis  encephalopathy  [5].  Also,  Al  accumulations  in  the  brain  have 
been  found  to  increase  in  patients  with  renal  failure.  Following  oral  exposure  to  this  metal,  retention  of  Al  was 
reported  in  the  hippocampus.  This  region  of  the  brain  is  rich  in  cholinergic  neurons.  These  and  other  observations 
suggest  that  Al  has  neurotoxic  properties.  Furthermore,  following  chronic  exposure,  Al  has  been  shown  to 
accumulate  in  all  regions  of  the  rat  brain.  There  is  evidence  of  a  relationship  between  the  high  levels  of  Al  and 
increased  risk  of  neurodegenerative  disorders,  including  AD  and  PD  [3]. 

Calcium 

Calcium  (Ca)  is  the  fifth  most  abundant  element  found  in  the  earth"s  crust.  Ca  occurs  most  commonly  in 
sedimentary  rocks  in  the  minerals  calcite,  dolomite,  and  gypsum.  Ca  is  essential  for  life  in  most  living  organisms, 
including  humans.  Ca  has  been  shown  to  be  elevated  in  the  brains  of  patients  with  AD:  it  concentrates  in  the 
amyloid  plaques.  These  plaques  are  one  of  the  neuropathological  hallmarks  of  AD.  However,  PSAPP  mice  (a 
mouse  model  of  AD)  appear  to  accumulate  less  Ca  in  their  plaques  compared  to  their  surrounding  brain  tissue  [6]. 

Chromium 

Chromium  (Cr)  is  also  found  in  the  earth' 's  crust  and  is  mined  as  chromite  ore.  Because  of  chromium"s  high 
corrosion  resistance  and  hardness,  it  is  added  to  nickel  to  create  stainless  steel:  this  is  the  most  common  application 
of  Cr.  It  is  also  used  in  dyes  and  pigments,  as  a  gasoline  additive,  and  in  the  tanning  of  leather.  The 
pathophysiological  role  of  Cr  has  not  been  elucidated.  However,  Cr  has  been  shown  to  moderately  increase  in  the 
parietal  cortex  of  patients  with  AD  when  compared  to  that  in  normal  brain  [7]. 

Copper 

Copper  (Cu),  like  most  elements,  is  mined  from  the  earth"s  crust.  Furthermore,  Cu  is  recycled.  Cu  is  commonly 
used  in  electrical  applications,  piping,  many  household  products  (e.g.,  sinks,  plumbing,  pots),  architectural 
applications,  coin-making,  and  chemical  applications.  Cu  has  been  found  to  be  vital  in  human  and  plant  life.  Cu  is 
introduced  into  the  body  mostly  via  food  intake.  Some  foods  that  are  high  in  Cu  are  oysters,  beef  or  lamb  liver, 
Brazil  nuts,  cocoa,  black  pepper,  lobster,  sunflower  seeds,  green  olives,  avocados,  and  wheat  bran.  The  brain 
contains  approximately  7.3%  of  the  total  body  Cu  supply. 

Cu  has  been  an  extensively  studied  metal  in  the  brain  [8].  In  a  normal  brain,  Cu  is  distributed  in  similar  ways  in 
the  central  and  subcortical  white  matter  of  the  cerebellum  of  both  young  and  old  brains;  however,  the  periphery  of 
the  dentate  nucleus  is  rich  in  Cu  [2].  The  higher  Cu  concentrations  are  mostly  found  in  the  cortex  and  hippocampus 
[8].  A  normal  aging  brain  has  been  shown  to  have  increased  Cu  levels,  especially  in  the  substantia  nigra  and  in 
some  cerebellar  regions  [4].  However,  when  the  levels  of  Cu  in  the  cortex  and  hippocampus  increase,  Wilson"s 
disease  is  said  to  result  [9].  In  AD,  there  is  an  abnormal  brain  Cu  distribution,  with  large  amounts  in  the  amyloid 
plaques  but  a  deficiency  in  the  neighboring  brain  tissue.  Other  researchers  also  reported  Cu  to  increase  in  the  senile 
plaques  [4].  These  findings  suggest  the  involvement  of  Cu  in  AD  is  multifactorial  and  complex. 

Iron 

Iron  (Fe)  is  one  of  the  most  common  metals  found  in  everyday  use.  Fe  makes  up  about  5%  of  the  earth"s  crust. 
The  earth"s  core  is  believed  to  consist  largely  of  an  iron-nickel  alloy  constituting  35%  of  the  mass  of  the  earth  as  a 
whole.  Because  of  this  distribution,  Fe  is  thought  to  be  the  most  abundant  element  on  earth.  Some  popular  food 
stuffs  rich  in  Fe  include  red  meat,  fish,  poultry,  beans,  vegetables,  black-eye  peas,  wheat,  and  cereals. 
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Fe  is  another  extensively  studied  metal  in  brain.  Fe  is  necessary  for  normal  brain  function  (e.g.,  in  learning  and 
memory)  [10].  In  a  normal  brain,  Fe  is  distributed  in  similar  ways  in  central  and  subcortical  white  matter  of  the 
cerebellum  of  both  young  and  old  brains;  in  the  cerebellar  cortex,  there  are  high  Fe  levels,  and  the  periphery  of  the 
dentate  nucleus  is  also  rich  in  Fe  [2].  Fe  appears  to  increase  rapidly  in  the  young  brain  and  then  remains  stable  until 
the  later  years;  increases  in  Fe  in  the  substantia  nigra  and  globus  pallidus  have  been  reported  [4].  Regions  where 
brain  Fe  is  high  include  substantia  nigra,  globus  pallidus,  red  nucleus,  caudate  nucleus,  and  the  putamen  [11]. 

Like  Cu  and  Ca,  Fe  has  been  shown  to  be  elevated  in  brains  of  patients  with  AD.  Elevated  Fe  is  found  in  the 
amyloid  plaques  [6].  Fe  is  also  moderately  increased  in  the  parietal  cortex  in  AD  patients  [7].  In  patients  with 
Attention  Deficit  Hyperactivity  Disorder  (ADHD),  Fe  has  been  shown  to  increase  in  the  substantia  nigra  [12]. 

Manganese 

Manganese  (Mn)  is  found  in  the  earth"s  crust  and  in  seawater.  It  is  often  found  with  Fe.  Some  of  the  more 
common  usages  of  Mn  are  in  steel,  aluminum  alloys,  alkaline  batteries,  coins,  and  pigments.  Some  popular  foods 
that  contain  Mn  include  tea,  spinach,  grains,  rice,  eggs,  nuts,  olive  oil,  green  beans,  and  fish. 

The  majority  of  the  human  body"s  Mn  is  found  in  liver  and  kidneys;  nevertheless,  Mn  is  an  important  element  for 
normal  brain  development  and  function  [13].  In  the  aging  brain,  Mn  has  been  shown  to  redistribute  itself  in 
different  brain  regions,  including  hypothalamus,  thalamus,  and  corpus  callosum  [4].  High  brain  Mn  concentrations 
are  related  to  PD.  There  has  been  much  research  indicating  that  human  striatum,  globus  pallidus,  and  substantia 
nigra  show  increases  in  Mn  levels  and  are  thought  to  be  target  sites  for  Mn  neurotoxicity.  The  globus  pallidus  and 
pituitary  glands  are  other  regions  where  Mn  preferentially  accumulates.  High  exposures  to  Mn  result  in  increases  in 
Mn  in  the  olfactory  epithelium  and  olfactory  bulb  [14].  It  is  noteworthy  that  Mn  levels  in  the  cerebellum  are  similar 
in  AD  and  normal  brains  while  Mn  increases  in  the  parietal  cortex  of  AD  brains  [7]. 

Magnesium 

Magnesium  (Mg)  is  the  8th  most  abundant  element  in  the  earth"s  crust  and  in  found  also  in  seawater.  Mg  is 
commonly  used  in  structural  building  materials,  automotive  parts,  electronic  devices,  aerospace  construction, 
photography,  and  fireworks.  Mg  is  an  essential  metal  in  human  and  plant  life  [13].  Human  Mg  deficiency  has  been 
linked  to  the  development  of  asthma,  ADHD,  and  osteoporosis.  Foods  that  are  high  in  Mg  include  spices,  nuts, 
cereals,  coffee,  cocoa,  tea,  and  vegetables.  In  neurodegenerative  disorders  such  as  AD  and  PD,  Mg  does  not  appear 
to  play  a  major  role.  For  example,  Mg  levels  in  the  cerebellum  and  parietal  cortex  do  not  differ  between  patients 
with  AD  and  normal  humans  [7] . 

Mercury 

Mercury  (Hg)  is  found  in  deposits  throughout  the  world,  mostly  as  cinnabar  (a  common  ore  of  Hg).  The  more 
common  uses  of  Hg  are  in  barometers,  thermometers,  dental  products  (e.g.,  fillings),  and  electrical  equipment  (e.g., 
computers,  telephones,  etc.)  [15].  Exposures  to  Hg  most  commonly  occur  from  the  consumption  of  marine  species 
[1].  Other  exposures  to  Hg  may  be  from  exhaust  of  coal -burning  power  plants,  cement  production,  batteries,  and 
gold  production.  The  role  of  Hg  in  neurological  diseases  such  as  AD  and  PD  has  been  controversial.  Hg  can  cross 
the  blood-brain  and  blood-placental  barriers.  It  is  then  retained  by  the  brain  for  years  [16].  Hg  tends  to  accumulate 
in  lipid-rich  regions  of  the  brain.  Human  exposure  to  Hg  usually  results  in  kidney  and  neurological  disorders  [15]. 

Potassium  &  Rubidium 

Potassium  (K)  occurs  in  nature  as  an  ionic  salt  and  is  essential  for  humans  and  other  animals.  It  is  found 
dissolved  in  seawater  and  in  other  minerals.  Some  common  food  sources  of  K  include  orange  juice,  potatoes, 
bananas,  avocados,  tomatoes,  broccoli,  apricots,  and  many  other  fruits. 

Rubidium  (Rb)  is  not  essential  for  humans,  but  is  readily  taken  up  by  the  body.  It  is  found  commonly  mixed  in 
with  other  elements,  especially  K.  Rb  is  found  in  some  plants  as  well.  Some  common  uses  of  Rb  are  in  fireworks, 
lasers,  chemical  applications,  and  electronic  transmission.  Rb  is  found  in  come  brain  tumors. 

Exposure  to  K  and  Rb  is  not  known  to  cause  neurodegenerative  disorders.  For  example,  the  concentrations  of  K 
and  Rb  are  similar  between  AD  and  control  brains  [1 ;  7]. 

Sodium 

Sodium  (Na)  is  an  essential  element  for  humans  and  other  animals.  It  is  found  in  nature  as  a  compound  only;  the 
most  common  forms  are  salt  deposits.  The  most  common  form  of  Na  that  we  know  and  use  is  sodium  chloride, 
table  salt.  It  does  have  many  other  industrial  uses  as  well.  Over-  or  under-exposure  to  Na  has  not  been  shown  to 
cause  neurological  disorders.  Nevertheless,  Na  has  been  shown  to  increase  every  brain  region  in  AD  patients  [1]. 

Zinc 
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Zinc  (Zn)  is  naturally  found  in  the  earth"s  crust  and  in  seawater.  It  is  normally  found  with  Fe  and  Cu  deposits. 
Zn  is  commonly  used  in  batteries,  production  of  brass,  bronze,  rubber,  in  pigments,  fire  retardant,  nuclear  weapons, 
automobile  engines,  and  agricultural  fungicides.  The  most  common  food  source  of  Zn  is  red  meats. 

Zn  is  another  essential  element  in  mammals.  Zn  is  distributed  in  similar  ways  in  the  central  and  subcortical  white 
matter  of  both  young  and  old  brains;  the  interior  of  the  dentate  nucleus  and  the  cerebellar  cortex  contain  high  levels 
of  Zn  [2].  The  regions  of  the  normal  brain  rich  in  Zn  include  the  hippocampus,  amygdala,  and  the  cortex.  As  the 
brain  ages,  its  Zn  distribution  changes,  just  in  the  regions  mentioned  above.  Nevertheless,  little  or  no  decrease  in 
brain  Zn  has  been  reported  in  aging  humans  or  rats  [4].  Zn  is  found  to  be  elevated  in  human  amyloid  plaques  while 
PSAPP  mice  (an  AD  mouse  model  that  does  not  show  neurodegeneration)  only  had  a  29%  increase  of  Zn  in  their 
plaques  compared  to  the  Zn  in  brain  tissue  surrounding  the  plaques  [6].  The  potential  role  of  Zn  as  a  cofactor  in  the 
pathogenesis  of  AD  was  strengthened  when  Zn  enrichment  was  found  in  senile  plaques  and  a  Zn  elevation  in  the 
neuropil  of  AD  patients  as  compared  to  those  in  control  individuals  [4]. 

Of  the  90  plus  naturally-occurring  elements,  26  are  known  to  be  essential  for  humans  and  animals  [17].  These 
consist  of  11  major  elements  or  macro-elements.  They  are:  C,  H,  O,  N,  S,  Ca,  P,  K,  Na,  Cl,  and  Mg.  Fifteen 
elements  are  known  as  trace  elements  or  micro -nutrients.  They  include:  Fe,  Zn,  Cu,  Mn,  Ni,  Co,  Mo,  Se,  Cr,  I,  F, 
Sn,  Si,  V,  and  As.  The  molecular  bases  for  the  essential  element  selection  and  rejection  have  not  been  elucidated, 
however.  Major  elements,  such  as  Na,  K,  Ca,  and  Mg,  are  required  for  bodily  functions  such  as  body  fluid  buffer, 
active  transport,  ionic  balance,  electrical  transmission,  tissue  development  as  well  as  the  composition  of  body  fluids 
and  structures.  Trace  elements  act  primarily  as  catalyst  in  enzyme  systems  in  cells  where  they  serve  a  wide  range  of 
functions  from  weak  ionic  effects  to  highly  specific  associations  such  as  metallo enzymes.  In  addition,  the  protein- 
metal  interactions  may  increase  the  stability  of  the  protein  moiety  to  metabolic  turnover. 

In  this  study,  we  investigated  30  elements  (including  electrolytes)  and  how  dietary  intake  on  a  life-time  basis 
would  affect  their  concentration  and  distribution  in  various  regions  of  the  rat  brain  (hypothalamus,  cerebellum,  pons 
and  medulla,  striatum,  mid-brain,  cerebral  cortex,  and  hippocampus)  and  discussed  their  health  implications.  The  30 
elements  of  interests  were:  Al,  As,  Ba,  Br,  Ca,  Cd,  Cl,  Co,  Cr,  Cu,  F,  Fe,  Hg,  I,  K,  La,  Mg,  Mn,  Mo,  Na,  Rb,  S,  Sb, 
Sc,  Se,  Si,  Sm,  Sr,  V,  and  Zn.  In  the  past,  research  has  primarily  focused  on  the  effects  of  different  elements  on  the 
brain  and  the  neurological  disorders  they  may  cause.  None  of  those  studies,  however,  have  examined 
simultaneously  as  many  elements  in  a  single  study  as  we  have  presented  here.  The  objective  of  this  paper  is  to 
characterize  the  levels  of  major  and  selected  trace  elements  in  various  regions  of  the  rat  brain  during  several  critical 
stages  of  development.  Because  during  the  first  8-9  days  of  age  postnatal,  the  rat  hippocampus  is  extremely  difficult 
to  visually  distinguish  from  the  overlying  cerebral  cortex,  we  had  included  the  hippocampus  tissue  together  with  the 
cerebral  cortex  for  analysis  in  the  case  of  5-day-old  tissue  samples.  The  four  postnatal  age  groups  of  rats  we  have 
studied  include  the  following: 

I.  5  days  old; 

II.  10  days  old; 

III.  22/23  days  old;  and 

IV.  120  days  old  (i.e.,  adult). 

While  humans  and  rats  are  not  identical  neurologically  and  physiologically,  there  are  more  similarities  between 
the  two  species  as  far  as  brain  structure  and  functions  are  concerned.  The  four  stages  of  the  postnatal  development 
of  the  rat  chosen  in  this  study  closely  parallel  the  postnatal  development  of  humans.  As  our  well  being  is  largely 
impacted  by  the  environment  we  are  living  in,  dietary  intake  is  definitely  a  major  factor  that  posts  a  long-term  effect 
on  our  health.  This  paper  provides  some  insight  into  how  our  diet  and  pollutants  in  our  diet  may  affect  our 
neurological  health.  Consequently,  our  experimental  results  may  allow  us  to  take  some  preventative  measures  in 
determining  how  best  to  optimize  and  gauge  our  dietary  intake  of  major  and  trace  elements  for  the  betterment  of  our 
health. 

2.  Experimental  Methods 

Wistar  rats  of  the  Portion  strain  (Animal  Breeding  Unit,  Carshalton,  Surrey,  U.K.)  bred  in  the  Institute  of 
Neurology,  University  of  London,  U.K.  were  used.  All  animals  were  kept  on  a  12-hour  light/12-hour  dark  cycle 
with  free  access  to  food  and  water.  Elemental  concentrations  in  the  rat  food  pellets  had  been  determined  (Table  1 
below). 
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Table  1.  Elemental  Concentration  of  rat  food  pellet. 


Element 

Concentration  (in  mg/g) 

Ca 

6.39  ±1.68 

Cl 

2.31  ±0.49 

Fe 

0.25  ±  0.05 

K 

5.36  ±0.95 

Mg 

1.17  ±  0.32 

Na 

1.48  ±0.32 

Concentration  (in  gg/g) 

Al 

98.06  ±  8.05 

Br 

10.57  ±2.26 

Co 

0.20  ±  0.04 

Cr 

1.50  ±0.44 

Cu 

7.21  ±2.12 

F 

ND(5.00) 

Hg 

ND  (0.25) 

I 

ND  (0.50) 

Mn 

49.34  ±8.65 

Mo 

2.80  ±0.45 

Se 

ND  (0.25) 

Rb 

12.73  ±2.61 

V 

ND  (0.50) 

Zn 

47.73  ±4.11 

Thirty  trace  and  major  elements  in  different  brain  regions  of  rats  at  the  four  different  age  groups  (i.e.,  I  through 
IV  as  defined  above)  were  analyzed  by  instrumental  neutron  activation  analysis  (INAA)  [18].  The  samples  were 
irradiated  by  thermal  neutrons  using  three  different  sets  of  conditions  depending  on  the  nuclear  characteristics  of  the 
elements  of  interest  [18].  Not  all  30  elements  were  reported  in  the  results  because  those  not  reported  herein  were 
either  below  the  detection  limits  of  our  INAA  technique  [18]  or  the  elements  were  not  retained  by  the  brain  tissue. 
All  values  listed  are  mean  ±  S.D.;  ND  =  not  detectable,  and  values  in  brackets  are  the  maximum  elemental 
concentrations  present  in  the  diet. 

As  already  noted  above,  in  age  group  I,  the  hippocampal  tissue  had  been  included  in  the  cerebral  cortex  and 
analyzed  as  such. 

All  the  standard  materials  and  samples  for  INAA  were  freeze -dried  and  pelleted.  Dried  samples  (rat  brain 
regions)  were  homogenized  using  a  Glen  Creston  polystyrene  ball-mill  and  pelleted.  Other  details  of  standard  and 
sample  preparations  were  as  described  previously  [18;  19]. 

Standards  and  samples  were  irradiated  in  a  100  KW  „Consort"  Reactor  Mark  II  at  the  Imperial  College  Reactor 
Centre.  Irradiated  standards  and  samples  were  then  analyzed  for  their  elemental  contents  by  gamma  ray 
spectrometry  using  various  Ge(Li)  detectors  and  ND6600  Multichannel  Analyzer  (Nuclear  Data  Inc.,  Schaumburg, 
Illinois,  USA).  Details  of  the  elemental  determinations  were  described  in  the  methodology  paper  by  Chan  et  al. 
(1983). 

All  laboratory  chemicals  used  were  of  analytical  grade  (BDH  Chemicals  Limited,  U.K.)  and  single  element 
standards  of  trace  elements  were  obtained  from  AAS  (Ventron  Division,  Limited,  U.K.);  other  elemental  standards 
for  F,  Cl,  Br,  and  P  were  obtained  from  Hopkin  and  Williams  Chemical  Ltd.,  U.K.  Water  for  solution  preparations 
and  feeding  was  double-distilled. 
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3.  Results 


Cerebellum 

The  cerebellum  is  the  “cauliflower- shaped”  region  of  the  brain  located  in  the  lower  part  of  the  brain  next  to  the 
brain  stem.  The  cerebellum  controls  movement,  balance,  and  coordination.  More  recent  evidence  suggests  that  it 
plays  roles  in  regulations  of  emotions  and  memory  and  learning.  Figure  1  shows  the  elemental  distributions  in  the 
cerebellum  during  rat  postnatal  development. 

It  can  be  noted  from  Figure  1  that  the  levels  of  different  elements  generally  show  a  decreasing  trend  as  the 
cerebellum  develops  and  reaches  adulthood:  this  trend  may  reflect  the  gradual  closing  of  the  blood-brain  barrier 
between  age  10  days  through  weanling  and  beyond.  However,  research  on  elemental  distributions  in  cerebellum 
during  development  has  been  minimal:  some  of  the  more  studied  elements  include  magnesium,  rubidium,  and 
sodium. 

Magnesium  levels  in  the  parietal  cortex  in  AD  patients  do  not  differ  from  those  in  normal  controls  [7].  Similarly, 
rubidium  levels  in  AD  brains  do  not  differ  from  those  in  control  brains  [1].  Consequently,  the  age-related  changes 
in  magnesium  and  rubidium  levels  in  the  cerebellum  may  not  be  have  good  predictive  value  of  neurological  diseases 
in  later  life.  On  the  other  hand,  the  sodium  levels  in  the  rat  cerebellum  decreases  during  the  latter  half  of  postnatal 
development  as  they  approach  the  adult  level.  On  the  other  hand,  sodium  levels  in  every  regions  of  the  human  brain 
are  significantly  higher  in  the  AD  patients  than  those  in  control  subjects  [1].  These  results  taken  together  suggest 
that  dietary  intake  of  sodium  may  have  some  relevance  in  the  causation  and/or  progression  of  AD. 

Cerebral  Cortex 
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Fig.  1.  Element  Concentrations  of  Rat  Cerebellum  at  Different  Development  Stages. 
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Fig.  2.  Element  Concentration  of  Rat  Cerebral-Cortex  at  Different  Developmental  Stages 

The  cerebral  cortex  is  responsible  for  all  intellectual  and  other  higher  functions  such  as  thinking,  voluntary 
movements,  language,  reasoning,  and  perception.  Figure  2  shows  the  elemental  distributions  in  the  cerebral  cortex 
during  postnatal  development  till  adulthood.  It  can  be  gleaned  from  the  results  shown  in  Figure  2  that  on  average, 
about  half  of  the  concentrations  of  different  elements  increase  with  increasing  age  while  about  half  of  the  levels  of 
the  other  elements  decrease  as  age  advances  toward  adulthood. 

Several  elements  in  the  human  cerebral  cortex  have  received  more  attention  recently;  they  are  described  in 
further  details  in  the  following: 

Even  though  the  physiological  role  of  aluminum  (Al)  in  brain  has  not  been  identified,  it  is  accumulated  in  the 
cerebral  cortex  of  the  rat  as  it  matures  to  adulthood:  presumably,  this  accumulation  reflects  dietary  intake  (Figure  2). 
Consequently,  increasing  brain  accumulation  of  Al,  which  is  known  to  be  neurotoxic,  could  induce  neurological 
problem(s)  in  the  long  term.  For  example,  Al  shows  significantly  higher  level  in  the  parietal  cortex  in  AD  patients 
compared  to  those  in  control  individuals  [7].  Similarly,  Al  accumulation  in  brain  and  other  tissues,  including  bone, 
liver,  and  kidney,  has  also  been  shown  to  accompany  renal  failure  [3]. 

Calcium  (Ca)  levels  in  the  rat  cerebral  cortex  do  not  markedly  change  during  postnatal  development  (Figure  2). 
On  the  other  hand,  one  of  the  pathological  hallmarks  of  Alzheimer' 's  disease  (AD)  is  the  accumulation  of  amyloid 
plaques  between  nerve  cells  (i.e.,  neurons)  in  different  parts  of  human  brain  including  cerebral  cortex.  Ca  levels  in 
human  amyloid  plaques  in  AD  are  elevated.  Consistent  with  the  latter  observation  is  the  finding  that,  in  PSAAP 
mice  (a  mouse  model  of  AD,  which  shows  plaques  but  little  neurodegeneration),  Ca  accumulation  in  their  plaques  is 
less  than  those  in  the  surrounding  brain  tissue  [6].  Consequently,  age-related  accumulation  of  Ca  in  this  brain  region 
could  potentially  serve  as  marker  for  the  likelihood  of  developing  neurodegeneration  as  age  advances. 

Chromium  (Cr)  shows  a  moderate  increase  in  parietal  cortex  of  AD  patients  compared  to  those  in  control  subjects 
[7].  In  rat  cerebral  cortex,  Cr  levels  increase  from  day  10  to  day  22/23  and  thereafter  decrease  as  adulthood  is 
reached  (Figure  2).  Nevertheless,  relations  between  brain  Cr  levels  and  neurological  diseases  have  not  been  firmly 
established. 


Page  486  of  531 


Copper  (Cu)  in  amyloid  plaques  in  AD  is  elevated  [4].  In  PSAAP  mice  (a  mouse  model  of  AD,  which  shows 
plaques  but  less  neurodegeneration)  Cu  accumulation  in  their  plaques  is  less  than  those  in  the  surrounding  brain 
tissue  [6].  Consequently,  increasing  Cu  accumulation  in  cerebral  cortex  as  age  advances  can  potentially  herald  the 
onset  of  neurological  problems. 

The  magnesium  (Mg)  levels  in  rat  cerebral  cortex  fluctuate  during  development  and  level  off  toward  adulthood 
(Figure  2).  Mg  levels  in  parietal  cortex  and  cerebellum  in  AD  patients  do  not  differ  from  those  in  control  subjects. 
Taken  together,  these  observations  suggest  Mg  to  be  more  predictor  of  neurological  problems  as  age  advances. 

Manganese  (Mn)  shows  significantly  higher  levels  in  the  parietal  cortex  of  AD  patients  compared  to  those  in 
control  subjects  [7].  However,  cerebellar  Mn  levels  in  AD  are  similar  to  those  in  control  subjects  [7].  Figure  2 
shows  that  Mn  levels  in  cerebral  cortex  of  the  rat  show  a  general  trend  of  decrease  as  its  age  increases.  Nonetheless, 
whether  this  is  a  generalized  trend  of  an  essential  element  remains  to  be  established. 

Potassium  (K)  concentrations  in  the  AD  brain  appear  to  be  similar  to  those  in  control  brain,  especially  in  the 
regions  examined  [1].  K  generally  shows  a  trend  of  age-related  increases  in  the  rat  cerebral  cortex  (Figure  2), 
perhaps  reflecting  increases  in  cellular  material  because  intracellular  K  is  higher  than  extracellular  K. 

Sodium  (Na)  levels  in  the  rat  cerebral  cortex  decrease  with  increasing  age  (Figure  2),  reflecting  the  closing  of  the 
blood-brain  barrier  thereby  excluding  the  entry  of  extracellular  Na  into  cells.  On  the  other  hand,  Na  levels  in  every 
region  of  the  AD  brain  are  higher  than  corresponding  levels  in  control  brain  [1].  Consequently,  age-related 
increases  in  brain  sodium  may  be  considered  at  least  a  partial  predictor  of  age-related  neurological  problems. 

Zinc  (Zn)  is  elevated  in  the  amyloid  plaques  in  AD  and  PSAPP  mice  (an  AD  mouse  model  that  shows  little  or  no 
neurodegeneration)  only  had  a  29%  increase  of  Zn  in  their  plaques  compared  to  the  Zn  in  brain  tissue  surrounding 
the  plaques  [6].  By  contrast,  Zn  levels  in  the  rat  cerebral  cortex  markedly  increase  between  age  22/23  days  and 
adulthood  suggesting  that  the  increase  is  associated  with  tissue  growth.  However,  the  increase  in  brain  Zn  during 
brain  aging  may  be  more  predictive  of  onset  and/or  presence  of  neurological  problems  [6]. 

Hippocampus 

The  hippocampus  is  located  deep  within  the  temporal  lobe  and  is  part  of  the  limbic  system.  This  brain  region  is 
responsible  for  learning  and  formation  of  long-term  memory.  Figure  3  shows  the  elemental  distributions  in  the  rat 
hippocampus  at  different  stages  during  postnatal  development.  We  discuss  below  the  elements  that  were 
emphasized  in  several  recent  reports. 

Following  oral  exposure  of  rats  to  aluminum  (Al),  its  retention  has  been  noted  in  the  hippocampus  as  occurs  in 
other  brain  regions:  rat  hippocampal  Al  levels  increase  markedly  after  10  days  of  age  (Figure  3).  In  patients  with 
PD,  elevated  Al  levels  have  been  found  in  several  brain  regions  including  the  hippocampus,  which  is  rich  in 
cholinergic  neurons  [3].  Consequently,  chronic  accumulation  of  Al  in  this  brain  region  over  a  life-span  may  lead  to 
the  development  of  neurodegenerative  diseases  such  as  PD  because  of  Al  is  known  to  be  neurotoxic. 

Potassium  (K)  levels  in  the  rat  hippocampus  increase  from  age  day  10  till  adulthood,  likely  reflecting  the  increase 
in  cellular  material  during  this  developmental  period  (Figure  3).  However,  in  the  brain  regions  examined  in  AD, 
brain  K  remains  essentially  unchanged  compared  with  corresponding  levels  in  control  subjects  [1],  suggesting  that 
brain  K  may  not  be  a  good  indicator  of  the  likelihood  of  developing  neurological  diseases  such  as  AD. 

Rubidium  (Rb)  levels  in  rat  hippocampus  fluctuate  between  age  day  10  and  adulthood  (Figure  3).  However,  Rb 
levels  in  the  AD  brain  do  not  differ  from  those  in  control  brain  [  1  ]  suggesting  that  Rb  levels  in  brain  may  not  be  a 
good  predictor  neurodegenerative  diseases  such  as  AD. 

Sodium  (Na)  levels  in  the  rat  hippocampus  decrease  between  ages  day  10  and  day  22/23  and  level  off  at 
adulthood  (Figure  3).  However,  Na  levels  in  every  part  of  the  AD  brain  examined  are  higher  than  corresponding 
levels  in  the  control  brain  [1]. 

Hypothalamus 

The  hypothalamus  is  located  at  the  interior  of  the  brain  under  the  thalamus.  It  controls  body  temperature, 
emotion,  thirst,  hunger,  appetite,  digestion,  and  sleep.  Figure  4  shows  the  elemental  distributions  in  rat 
hypothalamus  in  the  four  developmental  stages  we  have  investigated.  Even  though  this  brain  region  is  known  to 
accumulate  a  variety  of  metals,  their  precise  functional  significance  remains  to  be  fully  elucidated.  Only  few 
functionally  important  changes  during  development  can  be  commented  on  (Figure  4). 

The  electrolytes  in  this  rat  brain  region  show  fluctuating  trends  in  the  four  age  groups  investigated,  partly 
reflecting  the  closing  of  the  blood-brain  barrier.  For  example,  sodium  (Na)  and  chlorine  (Cl)  levels  significant 
decrease  in  the  first  22/23  days  postnatal.  On  the  other  hand  potassium  (K)  levels  markedly  increase  between  day  5 
and  day  10  postnatal. 
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Fig.  4.  Element  Concentrations  of  Rat  Hypothalamus  at  Different  Developmental  Stages 
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Although  they  show  some  age-related  fluctuations,  the  essential  trace  elements  copper  (Cu),  selenium  (Se),  and 
zinc  (Zn)  levels  in  the  rat  hypothalamus  significantly  increase  between  postnatal  age  5  and  adulthood  (Figure  4).  At 
present,  one  can  only  speculate  about  the  functional  significance  of  the  postnatal  increases  of  these  essential  metals 
in  the  rat  hypothalamus  because  of  the  absence  of  relevant  studies. 

Mid-Brain 

As  its  name  implies,  the  mid-brain  is  located  in  the  middle  of  the  brain  above  the  brain  stem.  The  mid-brain 
generally  acts  as  a  relay  station  for  both  sensory  and  spinal  cord  signals  and  passes  them  on  to  the  limbic  system  and 
the  cortical  areas.  Figure  5  shows  the  elemental  distributions  in  the  rat  mid-brain  in  the  four  age  groups  we  have 
investigated.  In  the  rat  mid-brain,  the  levels  of  both  major  and  trace  elements  studied  show  age-related  fluctuations 
that  differ  from  those  in  the  rat  cerebral  cortex  or  hippocampus.  However,  the  relevance  of  these  fluctuations  in  the 
mid-brain  to  neurological  diseases  remains  to  be  elucidated. 

Pons  and  Medulla 

Situated  immediately  below  the  mid-brain,  pons  and  medulla  constitute  the  brain  stem  which  is  connected  to  the 
spinal  cord.  Key  functions  of  pons  include  motor  control  and  sensory  analysis  and  signal  relay  between  the  mid- 
brain  and  spinal  cord.  The  medulla  is  known  to  be  responsible  for  maintaining  heart  and  breathing  rates.  Figure  6 
shows  the  elemental  distributions  in  the  rat  pons  and  medulla  in  the  four  age  groups  we  have  investigated.  Most 
major  and  trace  elements  show  age-related  decreases  in  their  levels  in  rat  pons  and  medulla  as  adulthood  approaches 
(Figure  6).  Overall,  the  accumulation  of  metals  in  this  brain  region  is  not  as  quantitatively  marked  as  those  in  the 
other  brain  regions,  possibly  due  to  the  fact  that  this  brain  region  is  full  of  fiber  tracts  rather  cell  bodies.  Moreover, 
the  physiological  significance  of  the  distributions  of  metals  in  this  brain  region  is  still  poorly  understood. 

Striatum 

The  striatum  constitutes  the  sub-cortical  parts  of  the  forebrain:  it  includes  the  substantia  nigra  and  the  globus 
pallidus.  Among  its  best  known  function  is  its  role  in  the  planning  and  modulation  of  movement  pathways  as  well 
as  roles  in  other  cognitive  functions.  Figure  7  shows  the  elemental  distributions  in  the  rat  striatum  in  the  four  age 
groups  we  have  investigated.  We  discuss  below  the  elements  that  were  emphasized  in  several  recent  reports. 
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Fig.  5.  Element  Concentrations  of  Rat  Mid-Brain  at  Different  Developmental  Stages 
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Fig.  6.  Element  Concentrations  of  Rat  Pons  &  Medulla  at  Different  Developmental  Stages 
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There  is  recent  evidence  that  iron  (Fe)  plays  a  role  in  the  consolidation  of  long-term  memory.  Brain  Fe 
distribution  overlaps  with  that  of  dopaminergic  neurons.  In  rodent  models  of  neurological  disorders,  Fe  levels  in  the 
substantia  nigra  and  basal  ganglia  increase.  Because  of  those  observations,  increased  striatal  Fe  has  been  suggested 
to  be  responsible  for  neuronal  cell.  Furthermore,  Fe  accumulates  in  brain  as  a  function  of  age  [19].  In  rat  striatum, 
Fe  levels  decrease  between  ages  10  days  and  22/23  days  and  level  off  thereafter  (Figure  7). 

Brain  Fe  is  highest  in  the  substantia  nigra,  globus  pallidus,  red  nucleus,  caudate  nucleus,  and  putamen.  Increased 
brain  Fe  has  been  found  in  several  neurodegenerative  disorders  although  it  has  not  been  defined  as  the  main  cause 
[21].  Fe  level  in  the  substantia  nigra  increases  in  PD  but  total  Fe  in  the  substantia  nigra  zona  reticulata  does  not 
change  in  either  PD  or  AD  [11].  Thus,  these  findings  suggest  that  over  accumulation  of  iron  in  striatum  may  lead  to 
development  of  neurodegenerative  diseases  such  as  PD  and  AD  and  sources  of  Fe  exposure  could  be  environmental 
and  dietary.  However,  it  remains  to  be  determined  if  the  elevated  Fe  levels  antedate  injury  of  pigmented  neurons  or 
constitute  a  consequence  of  neuronal  degeneration  because  the  increased  Fe  in  the  substantia  nigra  may  contribute  to 
oxidative  damage  to  neurons.  Nevertheless,  most  brain  Fe  is  stored  in  an  inactive  form  bound  to  intracellular 
ferritin,  which  is  thought  to  be  mainly  localized  in  microglia  and  oligodendroglia.  Clearly,  additional  studies  are 
needed  to  clearly  delineate  the  pathophysiological  role  of  Fe  in  neurodegenerative  diseases. 

Chronic  manganese  (Mn)  toxicity  in  human  induces  signs  and  symptoms  that  closely  resemble  those  noted  in  PD. 
There  is  a  great  deal  of  evidence  that  human  striatum,  globus  pallidus,  and  substantia  nigra  show  preferential 
increases  in  Mn  and  are  believed  to  be  the  primary  sites  associated  with  Mn  neurotoxicity.  Moreover,  exposure  to 
high  Mn  leads  to  elevation  of  Mn  levels  in  olfactory  epithelium  and  olfactory  bulb  [14]. 

In  every  region  of  the  brain  examined,  brain  sodium  (Na)  is  increased  in  AD  compared  with  corresponding  levels 
in  control  subjects  [1].  On  the  other  hand,  Na  levels  in  rat  striatum  decrease  during  postnatal  development.  Taken 
together,  these  observations  suggest  that  high  Na  accumulation  in  brain  may  lead  to  pathological  states  and  sources 
of  the  Na  could  be  environmental  and/or  dietary  in  origin. 

4.  Conclusions 

Our  results  strongly  suggest  that  the  metallomic  distributions  in  various  regions  of  the  rat  brain  change  markedly 
during  the  different  stages  of  postnatal  development.  In  conjunction  with  the  reports  in  the  literature,  our  findings 
also  suggest  that  brain  regional  metallomic  distribution  can  be  influenced  by  dietary  intake  of  metals  and  other 
elements  and  may  pathophysiological  implications  in  several  key  neurodegenerative  diseases  such  as  Parkinson"s 
disease  and  Alzheimef's  disease.  A  metallomic  distribution  model  of  various  regions  of  the  rat  brain  can  be 
developed  based  on  ours  experimental  results  and  statistical  analysis.  However,  much  remains  to  be  discovered 
regarding  the  brain  functional  significance  of  metallomic  distribution  during  development. 
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Abstract.  The  intake  and  concentration  of  metals  and  electrolytes  from  our  diet  are  believed  to  be  affecting  our  general  health, 
in  particular,  the  proper  functions  of  vital  organs.  For  example,  in  addition  to  other  genetic  and  environmental  factors, 
consuming  water  with  high  alkalinity  for  prolonged  time  is  suspected  to  lead  to  diseases  such  as  kidney  stone.  Evidence  has  been 
accumulating  that  excessive  metal  intakes  would  lead  to  organ  failures.  Once  absorbed,  minerals  and  electrolytes  can  travel 
freely  throughout  the  body,  and  distribute  at  key  organ  systems  such  as  the  brain,  lung,  kidney,  etc.  By  conducting  experiments 
with  animal  models  (e.g.,  rats),  it  is  possible  to  not  only  determine  where  the  organ  distribution  of  various  matrices  of  elements 
and  minerals  but  also  correlate  such  matrices  with  the  overall  physiological  and  behavioral  status  of  such  models.  In  this  study, 
information  matrices  of  30  different  elements  (including  heavy  metals  and  some  electrolytes)  and  their  distributions  in  various 
vital  organs  (e.g.,  brain,  lung,  kidney,  liver,  heart,  spleen,  and  uterus)  were  analyzed  as  a  function  of  normal  dietary  intake  at 
adulthood  (120  days  old).  An  elemental  (metal  and  electrolyte)  distribution  model  was  then  formulated  based  on  experimental 
results.  The  study  has  high  impact  to  our  understanding  of  how  environmental  health  would  affect  our  well  being.  This  study 
would  also  provide  insights  on  how  our  diet  would  affect  the  accumulations  of  unwanted  elements,  such  as  heavy  metals,  in  our 
vital  organs.  The  results  may  help  researchers  and  health  practitioner  to  identify  possible  links  between  daily  diet  (metals  and 
electrolytes)  and  diseases,  and  may  also  lead  to  a  better  understanding  of  diseases  associated  with  aging  such  as  Alzheimer’s  and 
Parkinson’s  diseases,  and  other  neurological  disorders. 

Key  words :  Brain  organs;  elements;  metals;  distributions;  diseases;  diet;  environment 


1.  Introduction 

Since  the  era  of  industrial  evolution,  living  conditions  for  humans  have  improved  drastically;  with  the 
consequence  of  better  living  conditions  and  less  physical  activities,  we  are  facing  other  aspects  of  health  issues  such 
as  obesity  and  hyper  immune  responses  (allergy).  In  modern  living,  considerable  attention  has  been  paid  to  dietary 
intake  or  supplement  due  to  health  concerns.  On  the  other  hand,  involuntarily  consumption  of  unwanted  chemicals 
and  preservatives  via  processed  food  and  polluted  water  is  also  possible.  One  such  example  would  be  consumption 
of  drinking  water  from  sources  that  are  laden  with  soluble  ions  and  heavy  metals.  This  occurs  quite  frequently  for 
those  that  are  living  in  rural  areas  with  no  treatment  system  for  their  drinking  water  in  developing  countries. 

Attempts  in  understanding  the  homeostasis  of  different  elements  in  brain  and  other  major  organs  fell  short 
significantly  due  to  the  vast  complexity  of  mechanisms  involved  [1].  What  cause  this  complexity  are  the  multiple 
factors  that  can  affect  the  dynamics  of  biological  functions.  For  examples,  an  element  of  different  compounds 
(chloride  versus  phosphate)  would  have  various  uptake  rates  by  the  body  [2]  and  the  uptake  rate  of  elements  in 
solution  by  the  digestive  system  was  proven  to  be  faster  than  in  food  stuffs.  Elemental  accumulations  are  not  solely 
related  to  exposure  but  likely  have  more  to  do  with  impairment  of  the  relevant  homeostasis  mechanism  [3]; 
conversely,  a  Cu  deficiency  in  mice  were  able  to  be  corrected  by  supplementation  of  Cu  in  drinking  water  [4], 
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2003).  In  addition,  larger  sample  size  is  needed  to  observe  statistical  significance  in  the  small  changes  over  life  time 
exposure  [5].  Such  constraint  creates  a  vast  obstacle  to  conduct  research,  and  the  hurdle  is  more  difficult  if  human 
subjects  are  involved. 

Currently,  the  understanding  of  homeostasis  mechanisms  for  accumulation  and  control  of  individual  element  in 
our  body  is  very  limited;  hence,  the  influence  of  elements  with  similar  properties  to  each  other  is  practically  non¬ 
existent.  However,  results  of  Zn  replacing  Cu  in  a  competitive  homeostasis  mechanism  in  rat’s  brain  were  recently 
reported  by  Maynard  et  al.  [5].  Therefore,  it  can  be  concluded  that  our  understanding  of  elemental  homeostasis 
mechanisms  in  our  body  systems  is  still  in  the  pioneer  stage,  and  opportunities  for  further  research  and  development 
are  widely  untouched. 

There  are  chemicals  (vitamins,  for  example)  and  elements  needed  to  maintain  proper  bodily  functions;  in  this 
study,  we  focused  on  the  essence  of  major  and  selected  trace  elements  and  how  these  elements  accumulated  in  our 
major  organs.  The  major  organs  are:  brain,  lung,  kidney,  liver,  heart,  spleen,  and  uterus.  Information  matrices  of 
30  different  elements  (including  heavy  metals  and  some  electrolytes)  that  were  fed  to  rats  as  part  of  a  regular  diet 
were  obtained  from  the  seven  major  organs  and  analyzed.  Measurements  of  the  elements  from  the  adult  rats  (120 
days  old)  were  compared  with  information  obtained  from  healthy  Japanese  males  [6].  A  linear  distribution  model 
based  on  experimental  results  and  statistical  analysis  for  the  elemental  accumulations  in  the  major  organs  was  then 
developed  to  relate  the  accumulation  ratio  between  rats  and  humans  taking  their  respective  regular  diet.  This  model 
can  be  used  as  an  assessment  tool  to  evaluate  elemental  homeostasis  in  different  organs  for  the  initial  screening  of 
potential  health  issues  and  diseases. 

2.  Experimental  Methods 

The  30  elements  included  in  this  study  were:  Al,  As,  Ba,  Br,  Ca,  Cd,  Cl,  Co,  Cr,  Cu,  F,  Fe,  Hg,  I,  K,  La,  Mg,  Mn, 
Mo,  Na,  Rb,  S,  Sb,  Sc,  Se,  Si,  Sm,  Sr,  V,  and  Zn.  Analytical  measurements  of  the  elements  from  the  organs  of  adult 
(120  days  old)  Wistar  rats  and  diet  pellets  that  were  used  to  feed  the  rats  were  reported  in  a  parallel  paper  at  the 
same  conference  (Metallomic  Distribution  in  Various  Regions  of  the  Brain  as  Influenced  by  Dietary  Intakes  and 
Their  Implications)  [7] . 

Information  from  the  organs  of  healthy  Japanese  males  was  abstracted  from  Katoh  et  al.  [5]:  The  data  were 
collected  from  64  persons  ranging  from  6  to  82  years  of  age  with  a  mean  age  of  42  years.  These  data  were  assumed 
to  be  representatives  of  healthy  humans  regardless  of  race. 

3.  Results 

3.1  Concentration  of  Elemental  Accumulations  with  Rats 

Table  1  tabulates  the  elemental  concentrations  obtained  from  7  major  organs  of  the  rats  which  were  feed  with 
regular  food  pellets.  Out  of  the  30  elements  of  interest,  only  20  elements  were  detectable  within  the  measurement 
limit  by  instrumental  neutron  activation  analysis  (INAA).  This  implies  that  either  the  concentrations  of  other 
(undetectable)  elements  were  too  low  for  the  analytical  method,  or  the  organs  did  not  retain  those  elements 
significantly.  The  latter  observation  was  especially  obvious  for  the  brain  that  showed  only  15  measureable 
elements. 

Table  1.  Measurements  of  elemental  concentrations  for  the  seven  major  organs.  Noted  that  concentrations  of  the  first  6  elements  (above  Al)  were 
in  mg/g  of  wet  weight  and  concentrations  of  the  rest  of  the  elements  were  in  pg/g  of  wet  weight.  ND  indicates  not  detectable. 


Element 

Liver 

Lung 

Heart 

Heart 

Spleen 

Uterus 

Brain 

Ca 

0.05 

0.08 

0.16 

0.04 

0.06 

0.09 

0.054 

Cl 

1.01 

1.39 

1.35 

0.8 

0.94 

1.99 

0.95 

Fe 

0.28 

0.14 

0.1 

0.1 

1.06 

0.09 

0.019 

K 

3.6 

1.94 

2.15 

2.39 

3.61 

2.35 

2.677 

Mg 

0.24 

0.11 

0.14 

0.15 

0.19 

0.22 

0.103 

Na 

0.64 

1.12 

1.11 

0.76 

0.57 

1.62 

0.957 

Page  494  of  531 


Al 

6.57 

3.98 

4.13 

5.68 

7.17 

3.43 

5.87 

Br 

4.67 

10.49 

7.76 

4.3 

4.96 

12.53 

1.91 

Co 

0.1 

0.03 

0.22 

0.04 

0.05 

0.02 

0.011 

Cr 

0.05 

0.06 

0.44 

0.33 

0.13 

0.58 

ND 

Cu 

5.41 

2.53 

10.39 

5.33 

3.79 

2.9 

2.1 

F 

2.41 

2.82 

4.95 

2.46 

ND 

6.15 

ND 

Hg 

0.12 

0.02 

0.53 

0.01 

0.2 

0 

ND 

I 

0.06 

0.05 

0.08 

0.05 

ND 

0.12 

ND 

Mn 

2.76 

0.2 

0.78 

0.32 

0.35 

0.19 

0.51 

Mo 

1.11 

1.37 

0.61 

0.28 

0.61 

0.71 

ND 

Rb 

16.27 

4.38 

5.04 

3.9 

5.79 

4.54 

2.66 

Se 

1.26 

0.4 

1.54 

0.39 

0.72 

0.44 

0.64 

V 

0.08 

0.01 

0.1 

0.07 

0.04 

0.08 

ND 

Zn 

41.13 

16.98 

24.56 

16.05 

17.34 

15.85 

10.79 

3.2  Elemental  Accumulations  in  Human  Organs 

Similar  to  Table  1,  Table  2  shows  the  corresponding  elemental  accumulations  in  6  organs  that  were  collected  by 
Katoh  et  al.  [6]  and  Magalhaes  et  al.  [8].  Discussions  with  the  brain  is  excluded  from  this  paper  but  are  discussed  in 
a  parallel  study  [7]  due  to  the  complexity  of  the  brain.  Katoh  et  al.  have  collected  21  elements  of  which  15  elements 
are  common  with  the  elements  from  the  rat’s  study  in  Table  1.  Magalhaes  et  al.  have  collected  fewer  elements, 
Table  2  only  shows  those  that  are  of  interested  in  this  study.  However,  Magalhaes  et  al.  also  compared  elemental 
concentrations  of  health  tissues  with  cancerous  tissues  of  the  same  individuals.  It  should  be  noted  that  samples  of 
Magalhaes  et  al.  were  expressed  in  pg/g  of  dry  weight  instead  of  wet  weight,  but  this  relative  weight  ratio  should  be 
compatible  for  both  approaches. 

Table  2.  Elemental  concentrations  of  various  organs  reported  by  Katoh  et  al.  [6],  the  concentrations  were  in  pg/g  of  wet  weight  except  as  noted. 
*The  elemental  concentrations  of  uterus  were  abstracted  from  Magalhaes  et  al.  [8]  with  samples  of  German  origin,  concentrations  were  reported 
as  pg/g  of  dry  weight,  except  otherwise  specified. 


Element 

Liver 

Lung 

Kidney 

Heart 

Spleen 

Uterus* 

Al 

11.3 

130 

10.4 

11.4 

12.5 

As(ng/g) 

300 

560 

400 

250 

380 

Br 

12.2 

48.6 

40.9 

16.7 

25.6 

19 

Ca 

208 

585 

529 

291 

335 

1590 

Cd 

16.7 

4 

237 

1.2 

5.4 

Cl(mg/g) 

4.2 

10.1 

10.9 

5 

6.7 

Co(ng/g) 

173 

123 

69 

70 

31 

Cs(ng/g) 

69 

88 

120 

113 

77 

Cu 

32.2 

14.2 

15.4 

18 

7.3 

4 

Fe 

837 

987 

430 

257 

1400 

59 

K(mg/g) 

9.4 

9.9 

10.4 

15.5 

15.9 

0.571 

La(ng/g) 

285 

311 

64 

32 

650 

Mg 

678 

565 

711 

1070 

729 

Mn 

5.58 

1.17 

4.92 

1.63 

0.82 
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Mo 

2.1 

1.37 

1.55 

0.69 

0.96 

Na(mg/g) 

4.2 

10.4 

10.7 

5.7 

5 

Rb 

22.3 

15.9 

17 

20.1 

22 

Sb(ng/g) 

52 

242 

69 

16 

49 

Sc(ng/g) 

5 

26 

4 

5 

5 

Se 

1.87 

1.3 

3.86 

1.37 

1.61 

Zn 

228 

62 

238 

126 

83 

3.3  Normalized  Elemental  Distribution  Ratio  in  Various  Organs  between  Rats  and  Humans 

A  normalized  elemental  distribution  ratio  can  be  obtained  by  dividing  the  concentration  of  an  element  in  a  rat 
organ  by  the  concentration  in  the  same  human  organ.  Figure  1  shows  the  normalized  elemental  distribution  ratio  for 
the  6  organs  and  15  elements  that  were  common  in  our  work  and  that  of  Katoh  et  al.  [6].  As  shown  in  Figure  1,  the 
elemental  distributions  (homeostasis)  for  the  15  elements  in  the  6  major  organs  fit  rather  consistently  in  general.  For 
examples,  Na  and  Cl  have  nearly  the  same  distribution  ratio  for  all  6  organs,  K  is  almost  as  consistent  except  for 
concentration  in  the  uterus.  This  may  be  due  to  diet  difference  between  rats  and  humans,  or  the  difference  is  a  factor 
of  physiological/genetic  origin;  a  proper  explanation  of  the  observation  remains  to  be  determined.  In  general,  Co 
has  the  widest  spread  in  distribution  among  organs. 

The  distribution  model  can  be  used  to  predict  physiological  development  from  rat  (animal)  to  human.  An 
abnormally  high  or  low  concentration  of  elements  in  organs  can  be  a  prelude  of  organ  malfunction  and/or  cancer 
development.  For  example,  samples  of  cancerous  uterus  tissue  had  1.6  time  the  concentration  of  K  when  compared 
with  that  of  normal  uterus  tissue  from  the  same  individual  [8].  However,  the  general  direction  of  increase  or 
decrease  in  elemental  concentration  of  cancerous  tissue  is  not  always  the  same  for  different  elements  and  organs. 
The  approach  of  normalized  distribution  ratios  developed  in  this  study  can  be  used  to  find  the  general  direction  of 
elemental  distribution  of  disease  tissues,  although  there  are  genetic  difference  between  rats  and  humans. 

4.  Conclusions 

A  normalized  elemental  distribution  model  is  developed  by  comparing  elemental  measurements  of  the  same 
organ  in  rats  and  humans.  The  model  can  be  used  to  predict  disease  tissues  based  on  their  elemental  measurements 
and  to  determine  how  well  we  can  absorb  elemental  nutrients  from  food.  This  study  has  high  impact  on  our 
understanding  of  how  environmental  health  would  affect  our  well  being.  This  study  would  provide  insights  on  how 
our  diet  would  affect  the  accumulations  of  unwanted  elements,  such  as  heavy  metals,  in  our  vital  organs.  The  results 
may  help  researchers  and  health  practitioner  to  identify  possible  links  between  daily  diet  (metals  and  electrolytes) 
and  diseases,  and  may  also  lead  to  a  better  understanding  of  diseases  associated  with  aging  such  as  Alzheimer’s  and 
Parkinson’s  diseases,  and  other  neurological  disorders. 
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1.  INTRODUCTION 

The  failure  of  cancer  chemotherapy  can  be  attributed  to  two  major  factors.  The  two  major  reasons  are 
toxic  effects  of  the  drugs  and  secondly  the  resistance  to  the  chemotherapeutic  agents  used  in  treatment  of 
cancer.  Although  the  design  of  the  drugs  is  to  get  rid  of  cancer  cells  in  the  body,  the  normal  cells  also  are 
exposed  to  the  anti-cancer  drugs  resulting  in  toxic  effects.  The  challenge  to  specifically  target  cancer  cells 
continues  and  there  have  been  many  drugs  designed  and  are  now  in  use  for  specific  cancer  cells.  Several 
drugs  like  herceptin  and  gleevec  are  in  use  in  clinic  with  improvement  of  survival  (1,  2).  It  becomes 
difficult  to  treat  cancer  when  few  of  the  cells  in  tumors  acquire  the  ability  to  adapt  to  the  toxic 
environment  of  the  drugs  that  are  used  in  chemotherapy  (3).  For  the  past  25  years,  many  laboratories 
around  the  world  have  focused  efforts  to  unravel  the  mechanisms  that  contribute  to  the  emergence  of 
resistance  in  cancer.  The  hallmark  of  resistance  was  the  discovery  of  P-glycoprotien  by  Dr.Victor  Ling. 
P-glycoprotein  was  found  to  be  an  active  transporter  and  pumped  drugs  outside  the  cells  (4).  This  in  turn 
decreased  the  intracellular  concentration  resulting  in  adaptation  of  cancer  cells  to  toxic  environment.  A 
downside  of  this  overexpression  of  P-glycoprotein  was  that  it  imparted  cross  resistance  to  several 
structurally  unrelated  drugs  (5).  Since  the  discovery  of  P-glycoprotein,  several  proteins  have  been 
identified  which  result  in  resistance  in  cancer.  Multidrug  Resistant  related  Proteins  (MRPs)  belong  to  a 
family  of  ATP  cassette  and  about  7  MRPs  have  been  identified  with  different  substrate  affinity  (6). 
Studies  in  our  laboratories  have  focused  on  many  drug  resistant  cells  from  leukemias,  sarcomas,  multiple 
myelomas,  breast  cancer  and  pancreatic  cancers.  We  have  shown  that  cisplatin  selected  methotrexate 
resistant  cells  have  a  66  kD  protein  that  in  its  unphosphorylated  state  fails  to  transport  the  drug  inside  the 
leukemia  cells  resulting  in  resistance.  Our  studies  with  sarcoma  cells  selected  for  resistance  to 
adriamycin  has  increased  expression  of  c-fos  indicating  a  role  of  this  transcription  factor  in  anthracycline 
resistance  (7,  8,  9).  Breast  cancer  cells  resistant  to  adriamycin  have  higher  expression  of  IL6  receptor 
(10).  Studies  with  dexamethasone  resistant  multiple  myeloma  cells  show  changes  in  intracellular 
signaling.  In  addition,  5-fluorouracil  resistant  pancreatic  cells  have  upregulated  levels  of  mitogenic  and 
drug  metabolizing  enzyme  (unpublished  data).  All  these  findings  from  our  laboratories  indicate  that 
resistance  is  a  complex  phenomenon  and  differs  in  different  cell  lines  as  well  as  mechanisms  vary 
depending  upon  the  exposure  of  the  chemotherapeutic  agent. 

We  took  a  different  approach  by  utilizing  nano  materials  to  study  the  effect  on  various  cancer  cells.  We 
exposed  Leukemia  cells  and  its  subline  that  was  selected  for  resistance  to  cisplatin  as  well  as  pancreatic 
cancer  cells  selected  for  resistance  to  5-fluorouracil  to  MnO  and  ZnO  nanoparticles  that  range  in  size 
from  30-80nm.  The  rationale  of  these  studies  was  to  determine  if  the  resistant  cells  differ  in  sensitivity  to 
nano  particles  and  if  the  cellular  signaling  pathways  were  altered  differentially  in  these  cell  limes.  The 
data  will  provide  us  with  information  if  cells  resistant  to  anticancer  drugs  respond  differently  to 
nanoparticles  so  that  we  may  design  drugs  using  nanomaterials  to  specifically  target  resistant  cells. 
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2. 


CYTOTOXICITY  OF  METALLIC  OXIDE  NANOPARTICLES  AND  THE 
UNDERLYING  SIGNALING  MECHANISMS 


Nanomaterials  have  been  increasingly  used  in  industrial  applications  (e.g.,  drug  delivery,  additives  to 
drugs  and  cosmetics).  Because  of  their  wide  use,  occupational  exposure  to  nanomaterials  and 
nanoparticles  may  pose  as  health  risks.  Recent  studies  have  suggested  that  exposure  to  nanoparticles  may 
induce  cytotoxic  effects  in  some  mammalian  cell  types  although  these  effects  have  not  been 
systematically  investigated  (11). 

In  a  series  of  studies,  we  tested  the  hypothesis  that  titanium  oxide,  magnesium  oxide,  and  zinc  oxide 
nanoparticles  exert  differential  cytotoxic  effects  on  human  astrocytoma  (U87)  cells  and  human  fibroblasts 
(HFF-1).  Because  the  effects  of  such  nanoparticles  on  human  neural  cells  are  unknown,  we  have 
determined  the  putative  cytotoxic  effects  of  these  nanoparticles  on  human  astrocytes-like  astrocytoma 
U87  cells  and  compared  their  effects  on  normal  human  fibroblasts.  We  found  that  TiC>2  micro-  and 
nanoparticles  induced  cell  death  on  both  human  cell  types  in  a  concentration-related  manner.  We  further 
noted  that  ZnO  nanoparticles  were  the  most  effective,  TiC>2  nanoparticles  the  second  most  effective,  and 
MgO  nanoparticles  the  least  effective  in  inducing  cell  death  in  U87  cells.  The  cell  death  mechanisms 
underlying  the  effects  of  TiC>2  micro-  and  nanoparticles  on  U87  cells  include  apoptosis,  necrosis,  and 
possibly  apoptosis-like  and  necrosis-like  cell  death  types.  Thus,  our  findings  may  have  toxicological  and 
other  pathophysiological  implications  on  exposure  of  humans  and  other  mammalian  species  to  metallic 
oxide  nanoparticles.  Furthermore,  we  are  of  the  opinion  that  these  mechanisms  can  be  exploited  to  induce 
the  death  of  cancer  cells  thereby  enhancing  the  process  of  new  anti-cancer  drug  discovery  (11). 


3.  CYTOTOXIC  EFFECTS  OF  METALLIC  OXIDE  NANOPARTICLES  AND  DRUG 
RESITANCE  MECHANISMS 


In  this  study,  we  determined  the  effect  of  MgO  nanoparticles  on  murine  leukemia  cancer  cells  (L1210 
cells)  and  a  subline  selected  for  resistance  to  cisplatin  (L/DDP  cells).  The  magnesium  oxide  nanoparticles 
showed  differential  effect  on  cell  survival  using  different  methods  of  assays.  We  further  determined  the 
effect  of  MgO  on  various  cells  signaling  proteins  (Akt,  p-Akt  and  Erk  and  p-Erk)  in  L1210  and  LDDP 
cells  after  magnesium  oxide  nanoparticles  treatment.  Our  results  indicate  that  the  treatment  with  MgO 
nanoparticles  increased  the  expression  of  p-Akt  and  decrease  the  expression  of  p-Erk  expression  in  L1210 
and  L1210/DDP  cells.  The  treatment  of  nanoparticles  showed  no  effect  on  the  level  of  expression  of  Erk 
and  Akt  in  these  cells  indicating  that  the  treatment  of  cells  with  MgO  nanoparticals  decreased  activation 
of  erk  pathway  but  increased  activation  of  Akt. 

We  further  examined  the  expression  of  downstream  transcription  factors  (c-fos  and  c-jun)  as  they  are 
altered  due  to  erk  signaling.  The  western  blot  analysis  shows  that  MgO  nanoparticles  decreased  the 
expression  of  c-fos  in  L1210  and  LDDP  cancer  cells.  Expression  of  c-jun  in  L1210  cells  showed  some 
increase  at  treatment  concentration  of  25pg/ml.  In  L1210/DDP  cells,  the  level  of  c-fos  reduced  in  dose- 
related  manner,  however  the  level  of  c-jun  was  undetected. 


4.  CONCLUSIONS 


We  have  presented  an  overview  of  our  studies  on  anti-cancer  drug  resitance  mechanisms  as  well  as 
ongoing  systematic  studies  of  the  nanotoxicity  of  metallic  oxide  nanoparticles.  Our  ongoing  studies  also 
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suggest  that  resistant  cancer  cells  may  exhibit  cross-resistant  to  the  cytotoxic  effects  of  metallic  oxide 
nanoparticles.  The  latter  effects  appear  to  be  mediated  by  cell  survival/proliferation  signaling 
mechanisms.  Clearly  these  are  important  and  novel  areas  of  anti-cancer  drug  discovery.  As  such  they 
merit  further  investigation.  We  are  currently  pursuing  these  exciting  new  domains  of  nanomedicine  and 
nanocancer  research. 
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Abstract  -  Highly  branched  polymers  and  nano  Au  particles 
sol-gel  were  used  to  construct  an  enzymatic  biofuel  cell 
system.  The  anode  and  cathode  of  this  biofuel  cell  modified 
with  the  sol-gel  were  characterized.  The  sol-gel  was  very 
effective  in  anchoring  and  stabilizing  lactate  dehydrogenase 
(LDH)  and  glutamate  dehydrogenase  (GDH),  the  enzymes 
used  in  the  biofuel  cell  system.  This  sol-gel  modification 
would  enable  further  enzymatic  biofuel  cell  development 
that  was  not  possible  due  to  rapid  deterioration  of  enzymes. 
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I.  Introduction 

Biochemical  fuel  cell  drew  heavy  popularity  among 
scientist  and  engineers  in  the  early  1960s  but  the 
commercialization  of  it  was  not  realized  due  to  technical 
difficulties  and  the  availability  of  inexpensive  petroleum 
energy  [1].  However,  the  recent  energy  crisis  and  the 
development  of  micro  circuitry  reiterate  the  interest  of 
biofuel  cell;  with  the  aid  of  nano  technology,  biofuel  cell 
has  leapfrogged  to  a  new  horizon.  In  addition  to  the 
conventional  application  of  waste  conversion  to  energy 
recycling  concept,  the  micro  and  nano  technology  enable 
the  concept  of  biofuel  cell  to  directly  extract  energy  from 
organisms  and  plants  [2,  3]. 

Our  research  group  has  started  a  biofuel  fuel  cell 
initiative,  the  objective  of  this  initiative  is  to  develop  a 
biofuel  cell  system  that  is  non-toxic  and  biological 
compatible  that  can  be  employed  in  a  natural  environment 
or  human  body.  In  this  paper,  we  are  reporting  an 
enzymatic  biofuel  cell  system  that  was  constructed  with 
electrodes  that  were  enhanced  with  polymer  and  nanogold 
particles  sol-gel  that  can  be  implanted  into  our  body  as 
power  source  for  biomedical  devices  such  as  pacemaker 
and  insulin  pump;  this  cell/battery  can  be  modified  to 
generate  electrical  current  to  be  utilized  at  point  of 
generation  or  stored  as  battery  power.  The  energy  source 
or  reactants  were  lactate,  a-ketoglutarate,  and  ammonia 
that  are  all  existing  in  our  body  or  as  parts  of  our  body 
fluid. 
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Senm- permeable 
Membrane 


Fig.  1  Atypical  fuel  cell,  products  of  fuel  cell  reactants  (fuel  oxidants) 
can  migrate  to  the  cathode  side  through  the  semi-permeable 
membrane  to  complete  the  overall  fuel  cell  reactions. 

For  our  biofuel  cell  system,  the  anodic  half-cell  was  made 
by  an  Au  electrode  coated  with  lactate  dehydrogenase 
(LDH)  submerged  in  solution  containing  lactate  and 
nicotinamide  adenine  dinucleotide  (NAD+),  thus,  for  the 
anodic  half  reaction: 

LDH 

Anode:  Lactate  +  NAD  — >  pyruvate  +  NADH  + 

2H+  +  2e"  (1) 

As  lactate  was  oxidized  into  pyruvate  in  the  anode,  the 
oxidative  potential,  E0’(25  °C),  would  be  0.19  V. 

The  cathodic  half-cell  was  made  by  a  glassy  carbon 
electrode  (GCE)  coated  with  glutamate  dehydrogenase 
(GDH)  submerged  in  solution  containing  ammonia  and 
reduced  nicotinamide  adenine  dinucleotide  (NADH),  the 
cathodic  half  reaction  was: 

Cathode:  a-Ketoglutarate  +  NH4+  +  NADH  +  H+  +  2e" 

GDH 

— >  glutamate  +  NAD  +  H20  (2) 

As  a-ketoglutarate  was  reduced  into  glutamate  in  the 
cathode,  it  generated  a  reductive  potential,  E0’(25  °C),  of  - 
0.14  V. 
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Combining  (1)  and  (2),  the  total  potential  for  the  fuel 
cell  with  the  anode  and  cathode  above  was  0.05  V  (per 
mole).  In  general,  biological  reactions  in  nature  do  not 
release  large  amount  of  energy  for  each  reaction  step  to 
preserve  energy  utilization  efficiency.  The  overall 
reaction  for  the  biofuel  cell  thus  was  [4] : 

+  LDH, GDH 

Lactate  +  a-ketoglutarate  +  NH4  - »  pyruvate  + 

glutamate  +  H+  +  H20  (3) 

Since  the  A  E0’  of  (3)  was  greater  than  0,  the  reaction  was 
spontaneous  and  would  proceed  as  written  from  left  to 
right. 


II.  MATERIALS  AND  METHOD 

A.  Electrodes 

Gold  electrode  had  diameter  of  0.3  cm  and  the  GCE 
had  diameter  of  0.35  cm,  they  both  were  purchased  from 
Tianjin  Aida  Heng  Sheng  Co,  Tianjin,  China. 

B.  Electrode  Preparation 

1).  Anode:  A  clean  gold  electrode  was  immersed  in 
0.1  M  cysteamine  solution  for  2  hours  in  the  dark,  then  it 
was  dipped  into  colloidal  gold  nanoparticles  for  24  hours, 
finally  the  electrode  was  dipped  into  LDH  solution  for  20 
hours  before  it  was  used  for  testing.  This  was  a  LDH-Au- 
cysteamine  (sol-gel)-modified  Au  anode  as  shown  in  Fig. 
2.  LDH  was  embedded  in  the  Au  sol-gel  matrix  which 
also  enabled  electronic  transfer. 


1-S-(CH2)2-NH2 
-s-(ch2)2-nh2 
-s-(ch2)2-nh2 


Electrode  wall 


Fig.  2.  A  gold  anode  modified  with  cysteamine,  Au  nanoparticles  (sol- 
gel)  and  LDH. 


2).  Cathode:  Several  combinations  of  ploymers 
(poly(amido  amine)  (PAMAM)  of  0th  and  2nd  generation, 
and  cysteamine)  with  GDH  and  nanogold  particles  were 
tested  for  their  efficacies  to  enable  the  reductive  reaction 
as  in  (2).  A  cleaned  glassy  carbon  electrode  was  first 
immersed  in  0.1  M  cysteamine  (or  PAMAM)  solution  in 
darkness.  The  resulting  monolayer-modified  electrode 
was  rinsed  thoroughly  with  twice -distilled  water  and 


soaked  in  water.  Then,  it  was  dipped  into  the  colloidal 
gold.  The  gold  colloid-cysteamine-modified  cathode  was 
dipped  into  the  1-glutamate  dehydrogenase  (GDH) 
solution  (pH  7.4)  (or  GDH  solution  containing  NADH). 
In  such  a  way,  a  GDH  (/NADH)  gold  colloid-cysteamine 
(or  PAMAM) -modified  glassy  carbon  cathode  was 
obtained. 

For  cathode  that  both  cysteamine  and  PAMAM  were 
coated,  the  cleaned  glassy  carbon  electrode  was  first 
immersed  in  cysteamine  solution,  the  resulting  electrode 
was  dipped  into  the  PAMAM  solution.  Then,  it  was 
dipped  into  the  colloidal  gold.  The  gold  colloid- 
cysteamine/PAMAM-modified  electrode  was  dipped  into 
the  GDH  solution  (pH  7.4)  (or  GDH  solution  containing 
NADH).  In  such  a  way,  a  GDH  (/NADH)  gold  colloid- 
cysteamine/PAMAM-modified  glassy  carbon  electrode 
was  obtained 

C.  Nanoparticles  Solution  Preparations 

Nanoparticles  Au  was  prepared  by  reacting  HAuC14 
with  citric  acid  [5]. 

All  chemical  reagents  used  in  this  study  were 
analytical  grade  or  the  highest  grade  available,  water  was 
double  deionized  distilled  water.  All  the  experiments 
were  carried  out  under  deoxygenated  condition  in  0.1  M 
phosphate  buffer  solution.  All  enzymes  and  biochemicals 
were  purchased  from  Sigma-Aldrich  Chemical  Co,  St. 
Louis,  MO,  USA. 

D.  Detections 

UV-VIS  spectrophotometry  was  carried  out  by  an 
Agilent  diodearray  spectrophotometer  when  needed; 
cyclic  voltammetry  was  conducted  by  using  a  Gamry  600 
Potentiostat.  Voltammetric  potential  was  measured 
against  a  saturated  chloride  electrode  (SCE). 

III.  RESULTS  AND  DISCUSSIONS 
A.  Anodic  half-cell 

Fig.  3  shows  the  oxidative  responses  of  the  modified 
Au  electrode  with  nanogold  particles,  cysteamine  and 
LDH.  As  seen,  the  characteristic  peak  of  lactate 
conversion  to  pyruvate  was  detected  at  250  mV  and  the 
accumulative  current  increased  linearly  with  added  lactate 
instantaneously  [6].  It  verified  that  the  modified  anodic 
electrode  was  functioning  as  expected  in  (1). 
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ketoglutarate  relationship  appeared  to  support  this 
observation. 


Fig.  3  Voltammetric  responses  of  an  Au  electrode  coated  with 
cysteamine,  Au  nanoparticles,  and  LDH  at  pH  8.  Responses  were 
stepwise  additions  of  lactate  at  l.Ox'4  mol/ml  with  linear  current 
increments  (oxidative  peaks)  at  250  mV. 


V,. 


ketoglutarate/mM 


B.  Cathodic  half-cell 

Fig.  4  is  the  UV-VIS  spectrum  PAMAM-Au-GDH 
coated  on  a  plastic  cuvette  that  has  no  absorption  in  the 
UV-VIS  range,  the  same  procedures  used  to  modify  the 
cathode.  As  shown,  PAM  AM  and  Au  have  strong 
absorption  at  278  and  548  nm,  respectively.  The  red  and 
blue  line  were  absorption  of  the  same  coating  after 
numerous  rinses  that  indicated  the  self-assembling  coating 
was  stable,  although  some  PAM  AM  were  released  after 
the  initial  rinses. 


Fig.  5  is  the  reductive  response  of  a  GCE  modified 
with  2nd  generation  of  PAMAMs,  nanogold  particles,  and 
GDH.  The  cathodic  half-cell  reaction  functioned  more 
efficiently  at  lower  concentrations  of  a-ketoglutarate  and 
became  less  effective  as  the  accumulative  concentration 
of  a-ketoglutarate  was  higher  (>0.025  mM).  This  could 
be  an  effect  of  saturation  that  the  nanoAu  particles  could 
only  support  a  limited  amount  of  GDH  for  the  conversion 
of  a-ketoglutarate  to  glutamate.  An  alternative 
explanation  is  that  some  PAMAM  was  released  by  the  Au 
sol-gel  that  resulted  in  less  conversion  efficiency.  This 
also  supported  the  general  speculation  that  the  Au  sol-gel 
can  only  support  a  finite  amount  of  “sorbed”  molecules 
per  unit  of  Au  molecule.  The  near  linear  current  versus  a- 


Fig.  5.  Reductive  responses  of  a  glassy  carbon  electrode  modified  with 
PAMAM,  Au  nanoparticle  and  GDH  measured  at  0.7  V  in  solution  of 
ammonia  and  a-ketoglutarate. 

For  the  PAMAM-Au-GDH  cathodic  half-cell,  the 
voltammetric  responses  of  (3)  at  0.7  V  was  shown  in  Fig. 
5,  the  total  current  measured  was  3.11  pA;  Considering 
the  electrode  surface  area  of  0.0962  cm2  (diameter  of  0.35 
cm)  and  reactants  consumption  of  0.3  mM,  thus  the 
current  density  (A/  cm2)  would  be  32.3  pA/  cm2  and  the 
current  generated  from  the  reaction  was  10.4  mA/M. 

It  was  observed  that  with  the  different  combinations 
of  cysteamine  and  PAMAMs,  polymers  with  the  most 
branched  structure  would  support  the  sol-gel  nanogold 
structure  with  the  most  efficiency  to  anchor  the  embedded 
enzymes  (LDH  and  GDH,  for  examples)  for  the 
enzymatic  conversions  [6].  In  addition,  this  sol-gel 
structure  on  the  electrodes  could  stabilize  the  enzymes 
that  normally  would  easily  decompose  rapidly  once  they 
are  in  diluted  solution,  which  was  the  case  in  our 
electrode  preparation.  Our  anode  and  cathode  performed 
repeatedly  consistently  as  they  were  freshly  prepared  even 
after  they  were  stored  submerged  in  distilled  water  at  4  °C 
for  2  weeks.  A  stable  and  cost  effective  enzymatic 
system  is  very  important  for  the  realization  of  using 
enzymatic  biofuel  cell  for  practical  applications. 

Our  biofuel  cell  system  would  generate  a  relative  low 
potential  (0.05V),  but  the  reactants  are  all  readily 
available  from  our  body  which  make  it  closer  to  reality 
for  the  system  to  be  implanted  into  our  body  as  power 
source  for  biomedical  devices.  If  this  concept  of 
enzymatic  biofuel  cell  is  to  be  employed  to  extract  energy 
from  an  industrial  waste  containing  high  energy  that  is 
normally  discarded,  such  as  waste  from  potato  processing 
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plant  and  cheese  factory,  then  a  more  potential  favorable 
redox  reaction  route  should  be  chosen [7].  For  example, 
the  current  cathode  with  (2)  has  a  negative  potential,  one 
can  choose  a  cathode  made  with  cytochrome  a  that  has  a 
positive  potential  as  shown  in  the  following: 

Cytochrome  a  +  Fe3+  +  e"  — ►  cytochrome  a  +  Fe2+ 

that  will  produce  a  reductive  potential  of  0.29  V  per  mole 
at  25  °C.  Another  half  reaction  with  high  positive 
potential: 

1/2  02  +  2H++  2e H20 

produces  a  reductive  potential  of  0.816  V  [8]. 

IV.  CONCLUSIONS 

An  enzymatic  biofuel  cell  system  consisted  of  anode 
and  cathode  modified  with  enzymes,  highly  branched 
polymers,  and  Au  nanoparticles  sol-gel  was  constructed 
and  characterized.  This  enzymatic  biofuel  cell  concept 
can  be  used  to  generate  power  for  various  biomedical  and 
environmental  applications,  depending  on  the  available 
energy  sources.  The  energy  sources  also  dictate  the 
enzyme  systems  to  be  used,  which  could  be  a  limiting 
factor  for  biofuel  cell.  The  highly  branched  polymers  and 
nanogold  particles  sol-gel  proved  to  be  effective  in 
anchoring  and  stabilizing  the  enzymes  used  in  the  biofuel 
cell  and  should  be  explored  further  in  the  development  of 
a  practical  biofuel  cell  system. 
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Abstract  -  The  intake  and  concentration  of  metals  and 
electrolytes  from  our  diet  are  believed  to  be  affecting  our 
general  health,  in  particular,  the  proper  functions  of  vital 
organs.  Other  than  genetic  factors  and  environmental 
stresses,  evidence  has  been  accumulating  that  excessive 
metal  intakes  would  lead  to  organ  failure.  Once  absorbed, 
minerals  and  electrolytes  can  travel  freely  within  the  body, 
and  deposit  at  key  organ  systems  such  as  the  brain,  lung, 
kidney,  etc.  By  conducting  experiments  with  an  established 
systematic  animal  model  (e.g.,  rats),  it  is  possible  to 
determine  the  elemental  distribution  matrices  within  the 
organs,  as  well  as  to  correlate  such  matrices  with  the  overall 
physiological  and  behavioral  status  of  the  animal.  In  this 
study,  rats  were  used  as  the  testing  animal  with  a  controlled 
diet  for  their  entire  lifetime.  Information  matrices  of  30 
different  elements  (including  heavy  metals  and  electrolytes) 
associated  with  their  distributions  in  various  vital  organs 
including  brain,  lung,  kidney,  liver,  heart,  spleen,  and  uterus 
were  analyzed  as  a  function  of  age  at  5,  10,  23,  and  120  days 
old.  In  addition,  how  excessive  Mn  intakes  would  affect  the 
accumulation  of  other  elements  was  also  studied.  It  was 
found  that  the  excessive  Mn  intakes  would  greatly  affect  the 
accumulation  of  most  metals  and  electrolytes  from  the  intake 
diet,  although  the  diet  only  contained  trace  amount  of  some 
of  these  elements.  For  example,  excessive  amount  of  Hg  was 
found  in  kidney  issue  compared  to  control  sample.  This 
multidimensional  analysis  of  elemental  accumulations  in 
major  organs  provided  groundbreaking  information  in  our 
understanding  of  how  dietary  intakes,  including  pollutants, 
can  directly  affect  our  physiological  condition  and  general 
health. 

Keywords  -  Metals,  electrolytes,  distribution,  Mn,  diet, 
accumulation. 

I.  INTRODUCTION 

In  the  contemporary  society,  considerable  attention 
has  been  paid  to  dietary  intake  or  supplement  due  to 
health  concerns.  On  the  other  hand,  involuntarily 
consumption  of  unwanted  chemicals  and  preservatives  via 
processed  foods  and  polluted  water  is  also  inevitable 
depending  on  the  lifestyle  and  available  resources  of 
individuals;  one  such  example  would  be  consumption  of 
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drinking  water  source  that  is  laden  with  soluble  ions  and 
heavy  metals.  This  occurs  quite  frequently  for  those  that 
are  living  in  rural  areas  with  no  treatment  system  for  their 
drinking  water  in  developing  countries. 

Attempts  in  understanding  the  homeostasis  of 
different  elements  in  brain  and  other  major  organs  fell 
short  significantly  due  to  the  vast  complexity  of  the 
mechanisms  involved  [1].  What  cause  this  complexity 
are  the  multiple  factors  that  can  affect  the  dynamics  of 
biological  functions,  and  often  these  factors  are  not 
apparent  until  the  advanced  age  of  an  animal.  In  addition, 
larger  sample  size  is  needed  to  observe  statistical 
significance  in  the  small  changes  over  life  time  exposure 
[2].  Such  constraint  creates  a  vast  obstacle  to  conduct 
research,  and  the  hurdle  is  more  difficult  if  human  subject 
is  involved,  making  studying  such  factors  prohibitively 
costly. 

There  are  chemicals  and  elements  that  we  need  to 
maintain  proper  bodily  functions:  in  this  study,  we 
focused  on  the  essence  of  major  and  selected  trace 
elements  and  how  these  elements  accumulated  in  our 
major  organs.  The  major  organs  are:  brain,  lung,  kidney, 
liver,  heart,  spleen,  and  uterus.  Information  matrices  of 
30  different  elements  (including  heavy  metals  and  some 
electrolytes)  that  were  fed  to  rats  as  part  of  a  regular  diet 
were  obtained  from  the  seven  major  organs  and  analyzed. 
The  30  elements  of  interests  were:  Al,  As,  Ba,  Br,  Ca,  Cd, 
Cl,  Co,  Cr,  Cu,  F,  Fe,  Hg,  I,  K,  La,  Mg,  Mn,  Mo,  Na,  Rb, 
S,  Sb,  Sc,  Se,  Si,  Sm,  Sr,  V,  and  Zn. 

Also,  we  investigated  how  intake  of  an  excessive 
contaminant,  such  as  a  heavy  metal  ion  (Mn2+)  in  this 
study,  would  alter  the  accumulation  of  all  other  elements 
and  electrolytes  in  the  vital  organs  over  the  lifetime  of  the 
rats,  that  would  directly  affect  the  proper  functions  of 
these  organs. 

II.  EXPERIMENTATION 

Wistar  rats  of  the  Portion  strain  (Animal  Breeding 
Unit,  Carshalton,  Surrey,  U.K.)  bred  in  the  Institute  of 
Neurology,  University  of  London,  U.K.  were  used.  All 
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animals  were  kept  on  a  12-hour  light/1 2-hour  dark  cycle 
with  free  access  to  food  and  water.  Elemental 
concentrations  in  the  rat  food  pellets  have  been 
determined  as  shown  in  Table  1  below. 


TABLE  I.  Elemental  Concentration  of  rat  food  pellet. 


Element 

Concentration  (in  mg/g) 

Ca 

6.39  ±1.68 

Cl 

2.31  ±0.49 

Fe 

0.25  ±  0.05 

K 

5.36  ±0.95 

Mg 

1.17  ±0.32 

Na 

1.48  ±0.32 

Concentration  (in  gg/g) 

Al 

98.06  ±  8.05 

Br 

10.57  ±2.26 

Co 

0.20  ±  0.04 

Cr 

1.50  ±0.44 

Cu 

7.21  ±2.12 

F 

ND(5.00) 

Hg 

ND  (0.25) 

I 

ND  (0.50) 

Mn 

49.34  ±8.65 

Mo 

2.80  ±0.45 

Se 

ND  (0.25) 

Rb 

12.73  ±2.61 

V 

ND  (0.50) 

Zn 

47.73  ±4.11 

Thirty  trace  and  major  elements  in  different  organs  of 
rats  at  four  different  age  groups  (5,  10,  23,  and  120  days) 
were  analyzed  by  instrumental  neutron  activation  analysis 
(INAA)  [3].  The  samples  were  irradiated  by  thermal 
neutrons  using  three  different  sets  of  conditions 
depending  on  the  nuclear  characteristics  of  the  elements 
of  interest.  Not  all  30  elements  were  reported  in  the 
results  because  those  not  reported  herein  were  either 
below  the  detection  limits  of  the  INAA  technique  [3]  or 
the  element  concentrations  were  not  high  enough  in  the 
samples  (food  pellets  or  organ)  for  detection. 

All  the  standard  materials  and  samples  for  INAA  were 
freeze-dried  and  pelleted.  Dried  samples  were 
homogenized  using  a  Glen  Creston  polystyrene  ball-mill 
and  pelleted.  Other  details  of  standard  and  sample 
preparations  were  as  described  previously  [3,  4]. 

Standards  and  samples  were  irradiated  in  a  100  KW 
‘Consort’  Reactor  Mark  II  at  the  Imperial  College  Reactor 


Centre.  Irradiated  standards  and  samples  were  then 
analyzed  for  their  elemental  contents  by  gamma  ray 
spectrometry  using  various  Ge  (Li)  detectors  and  ND 
6600  Multichannel  Analyzer  (Nuclear  Data  Inc., 
Schaumburg,  Illinois,  USA).  Details  of  the  elemental 
determinations  were  described  in  the  methodology  paper 
by  Chan  et  al.  [3]. 

All  laboratory  chemicals  used  were  of  analytical  grade 
(BDH  Chemicals  Limited,  U.K.)  and  single  element 
standards  of  trace  elements  were  obtained  from  AAS 
(Ventron  Division,  Limited,  U.K.);  other  elemental 
standards  for  L,  Cl,  Br,  and  P  were  obtained  from  Hopkin 
and  Williams  Chemical  Ltd.,  U.K.  Water  for  solution 
preparations  and  feeding  was  double-distilled. 

The  three  concentrations  of  Mn2+  fed  to  the  rats  via 
drinking  water  were  1,10,  and  20  mg/ml. 

III.  RESULTS  AND  DISCUSSIONS 

As  shown  in  Lig.  1,  there  were  only  20  elements 
detectable  from  the  kidney  tissue  of  the  rats  out  of  the  30 
elements  of  interest.  The  rest  of  elements  were  either 
under  the  detectable  limit  of  INAA,  or  the  kidney  did  not 
retain  these  elements:  As,  Ba,  Cd,  La,  S,  Sb,  Sc,  Si,  Sm, 
Sr.  Notes  that  some  of  the  undetectable  elements  from 
the  food  pellets  showed  up  in  the  kidney  tissue  (such  as  L, 
Hg,  I,  Se,  and  V)  which  indicated  that  the  elements  would 
accumulate  with  time  and  the  amounts  would  also  depend 
on  the  organs,  thus  their  functions. 

Some  elements  increased  with  time  (age);  many 
elements  would  decrease  slightly  or  level  off  with  age, 
these  variations  were  also  organ  dependent. 
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Fig.l  Elemental  concentrations  of  rats’  kidney  at  different 
developmental  stages  of  5,  10,  23,  and  120  days,  (a)  The  weight  of 
elements  in  pg/g  of  organ  in  wet  weight;  (b).  weight  of  elements  in  mg/g 
of  organ  in  wet  weight. 
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It  was  found  that  Mn2+  intake  would  definitely  affect 
the  retention  of  other  elements  in  the  major  organs.  Fig.  2 
shows  the  fractional  changes  of  different  elements  in  the 
kidney  tissue  when  3  different  concentrations  of  Mn2+ 
were  fed  to  120  days  old  rats,  as  compared  to  the  120 
days  old  control  without  excessive  Mn2+.  The  intake  of 
excessive  Mn  increased  the  accumulation  of  most  heavy 
metals.  More  noticeably,  it  also  increased  the 
accumulation  of  I  and  Hg,  and  decreased  the  retention  of 
Se  and  V,  all  of  which  were  not  detectable  in  the  food 
pellet.  Therefore,  if  we  consume  an  excessive  amount  of 
an  heavy  metal  (for  example,  Mn)  for  a  lengthy  period,  it 
is  likely  that  the  total  toxicity  would  be  synergistically 
increase  due  to  increased  accumulation  of  other  heavy 
metals  induced  by  the  excessive  Mn.  This  synergistic 
effect  has  been  observed  consistently  in  all  the  organs  we 
studied. 


IV.  CONCLUSIONS 

Retention  of  elements  from  diet  intake  was  found  to 
be  age  and  organ  dependent.  Excessive  intake  of  a  heavy 
metal  can  induce  synergistic  toxicity  due  to  increased 
accumulation  of  other  heavy  metals  presented  in  the  diet, 
although  the  concentrations  of  these  metals  may  be  at 
trace  level. 
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Fig.  2.  Different  concentration  treatments  with  Mn2+  in  water  with  %  changes  in  adult  kidney  relative  to  control  adult  level:  A). Life-long  treatment  with  1 

mg/ml  of  Mn2+;  B.)  10  mg/ml  of  Mn2+;  C.)  20  mg/ml  of  Mn2+. 
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Abstract:  We  previously  reported  the  development  of  a  biosensor  platform  that  is  capable  of 
measuring  biometabolites  and  environmental  sensitive  species,  such  as  peroxide  and  nitrate/nitrate,  to 
concentrations  in  the  order  of  ppb  (parts  per  billion)  or  lower.  In  this  investigation,  we  modified  our 
platform  with  dendrimers  to  enhance  its  performance.  Zero  and  second  generation  of  dendrimers  were 
coated  on  the  surface  of  a  carbon  glassy  electrode  which  was  then  modified  with  1-glutamate 
dehydrogenase  (GDH)  and  a-keto  glutarate.  The  resulting  electrode  was  tested  with  ammonium 
solutions,  concentrations  ranged  from  2  to  300  nM  at  pH  7.4;  the  results  were  satisfactory. 
Measurements  at  lower  concentrations  had  better  resolution  than  at  higher  concentrations  and  it  is 
believed  that  the  measurement  limit  can  be  lower  than  2  nM.  This  biosensor  platform  was  proven  to  be 
versatile  and  can  be  employed  as  a  platform  for  ultrasensitive  detecting  devices  in  many  biomedical 
and  environmental  applications.  Copyright  ©  2011  IFSA. 

Keywords:  Biosensor,  GDH,  Ammonium,  Glassy  carbon  electrode,  PAMAM. 


1.  Introduction 

In  the  past  several  years,  our  research  group  has  been  developing  a  platform  for  biosensor  construction 
that  is  capable  of  detecting  target  species  that  are  in  nano  and  subnanomolar  levels  [1,2].  This  sensor 
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platform  is  based  on  an  Au  electrode  modified  with  sol-gel  Au  colloid  and  biocatalyst  that  is  specific 
to  the  chemical  species  to  be  detected,  the  Au  colloid  comprises  nano-size  Au  particles  and  cysteamine 
which  binds  strongly  on  the  surface  of  the  metal  electrode  and  provide  stable  linkage  with  the  nano 
gold  particles  and  biocatalysts.  Thus  far,  biometabolites  and  environmental  sensitive  chemicals  such  as 
nitrite/nitrate  and  peroxide  were  successfully  detected  at  level  of  nM  and  below  with  this  sensor 
platform. 

In  this  study,  we  were  developing  another  biosensor  platform  that  can  be  used  to  measure 
biometabolites  and  environmental  sensitive  chemicals  based  on  a  non-metal  glassy  carbon  electrode 
modified  with  sol-gel  Au  colloid  that  is  similar  to  the  sensor  platform  mentioned  previously,  except  the 
binding  polymer  would  be  a  dendrimer  (polyamidoamine,  PAMAM)  instead  of  cysteamine.  A 
dendrimer  was  chosen  to  be  the  linker  between  the  electrode  and  Au  particles/catalysts  because  of  its 
low  toxicity  and  highly  branched  structure  that  presumably  can  provide  more  anchoring  sites  for  the 
Au  particles  and  catalysts.  Thus,  the  dendrimer  can  enhance  the  performance  of  the  sensor  platform. 
However,  PAMAM  does  not  have  an  S  atom  in  the  molecular  structure  and  may  not  bind  as  well  on 
the  electrode’s  wall  thus  would  be  less  durable.  Performance  of  using  cysteamine  and  dendrimer  as  a 
linker  for  the  same  sensor  platform  was  compared  in  this  study  and  the  stability  and  durability  of  the 
senor  would  be  investigated.  Ammonia,  which  is  a  major  biometabolite  and  environmental  sensitive 
chemical,  was  used  as  the  surrogate  chemical  to  be  tested  by  this  sensor  platform  [3]. 


1.1.  Theory 

As  shown  in  the  following  Equation  (1),  glutamate  and  nicotinamide  adenine  dinucleotide  (NAD+)  can 
be  hydrolyzed  to  form  a-keto  glutarate,  P-nicotinamide  adenine  dinucleotide  reduced  (NADH),  and 
ammonium  ion  with  the  enzyme,  glutamate  dehydrogenase  (GDH).  The  equilibrium  constant  is  in 
favor  of  the  formation  of  glutamate  and  thus  the  reverse  reaction  is  faster  kinetically  [4], 
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For  many  biological  reactions,  the  end  product  or  metabolite  is  ammonium;  accumulation  of  a  high 
concentration  of  ammonium  is  toxic  to  the  body.  Environmentally,  ammonium  is  a  byproduct  of  many 
industrial  processes,  uncontrolled  discharges  of  ammonium  will  lead  to  harmful  consequence  to  the 
environment.  Our  goal  here  was  to  develop  a  sensor  device  that  is  nontoxic  and  can  measure  the 
ammonium  ion  at  the  lowest  concentration  possible;  its  utilities  can  be  found  in  many  biomedical  or 
environmental  applications.  Hence,  it  is  the  reverse  reaction  as  shown  above  (glutamate  formation) 
that  we  utilized  in  this  study;  however,  the  same  sensor  can  be  used  to  detect  a-keto  glutarate  or 
glutamate  with  a  slight  alternation  of  the  measuring  approach.  Measurements  of  these  chemical  species 
mentioned  above  that  were  reported  in  literature  were  mostly  in  the  range  of  mM  [5-7],  It  should  be 
noted  that  the  motivation  of  this  study  was  to  develop  a  sensor  that  can  measure  low  concentration  of 
the  target  species,  more  specifically,  it  is  to  relate  the  current  measurements  exerted  by  the  reacting 
species  in  a  solution  (with  characteristics  peaks  of  oxidation  and/or  reduction)  by  cyclic  voltammetry 
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to  concentrations  of  the  species,  rather  than  to  determine  the  reaction  kinetics  of  the  measuring  species 
of  which  in  many  instances  are  already  known. 


2.  Materials  and  Methods 

2.1.  Materials 

L-glutamic  dehydrogenase  (GDH,  from  bovine  liver,  solution  in  50  %  glycerol)  was  purchased  from 
Sigma-Aldrich  (St.  Louis,  MO,  USA),  the  concentration  of  GDH  was  28  mg  protein/mL,  and 
46  units/mg.  Cysteamine,  polyamidoamine  dendrimer  generation  0  (PAMAM  O),  polyamidoamine 
dendrimer  generation  2nd  (PAMAM_2),  a-keto  glutarate,  P-nicotinamide  adenine  dinucleotide  reduced 
disodium  salt  hydrate  (NADH),  AuCfHCl-  4H2O  (Au  %>  48  %)  and  Na3citrate  were  purchased  from 
Sigma.  All  the  other  chemicals  were  of  analytical  grade  or  highest  grade  available. 


2.2.  Electrode  Preparation 

The  cleaned  glassy  carbon  electrode  (GCE)  was  first  immersed  in  0.1  M  cysteamine  solution  in 
darkness.  The  resulting  monolayer-modified  electrode  was  rinsed  thoroughly  with  twice-distilled  water 
and  soaked  in  distilled  water.  Then,  it  was  dipped  into  the  colloidal  gold.  The  gold  colloid-cysteamine- 
modified  electrode  was  dipped  into  the  1-glutamate  dehydrogenase  (GDH)  solution  (pH  7.4)  (or  GDH 
solution  containing  NADH).  In  such  a  way,  a  GDH  (or  GDH/NADH)  gold  colloid-cysteamine- 
modified  glassy  carbon  electrode  was  obtained  (GCE  cysteamine-Au-GDH  or  cysteamine-Au- 
GDH/NADH). 

For  electrode  that  was  modified  by  PAMAM  only,  either  PAMAM  _0  or  PAMAM  2  was  used  in 
place  of  cysteamine,  these  resulting  electrodes  were  termed  GCE  PAMAM  _0  or  PAMAM_2-Au- 
GDH  or  GDH/NADH.  In  cases  that  both  cysteamine  and  PAMAM  were  coated  onto  electrodes,  the 
cleaned  glassy  carbon  electrode  was  first  immersed  in  cysteamine  solution,  the  resulting  electrode  was 
dipped  into  the  PAMAM  solution  (PAMAM  _0  or  PAMAM2).  The  gold  colloid- 
cysteamine/PAMAM-modified  electrode  was  dipped  into  the  GDH  solution  (pH  7.4)  (or  GDH  solution 
containing  NADH).  In  such  a  way,  a  GDH  (or  GDH/NADH)  gold  colloid-cysteamine/PAMAM- 
modified  glassy  carbon  electrode  was  obtained  (GCE  cysteamine/PAMAM  _0  or  PAMAM_2-Au- 
GDH  or  GDH/NADH). 


2.3.  Nanoparticles  Solution  Preparations 

Nanoparticles  Au  was  prepared  by  reacting  HAuCf  with  citric  acid  [2], 

2.4.  Detections 

UV-VIS  spectrophotometry  was  carried  out  by  an  Agilent  diode-array  spectrophotometer  (Agilent 
Model  8453);  cyclic  voltammetry  was  conducted  by  using  a  Gamry  600  Potentiostat.  Voltammetric 
potential  was  measured  against  a  saturated  chloride  electrode  (SCE).  The  experimental  setup  of  the 
cyclic  voltammetry  is  shown  in  Fig.  1. 
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Fig.  1.  Cyclic  voltammetry  and  the  sensor  measurement  cell. 


2.5.  Experimental  Procedures 

For  all  experiments,  the  measurement  cell  contained  5  mL  of  2.5  mM  of  NADH,  then  a  combined 
solution  of  a-keto  glutarate  and  NH4+ was  injected  into  the  reaction  cell  for  measurement  periodically, 
at  60  second  intervals;  injection  volume  varied  from  10  to  1500  pL  depending  on  the  concentration 
required.  The  concentration  of  a-keto  glutarate  solution  was  lxlO6  mol/L,  NH4+was  lxl O'6  mol/L.  All 
experiments  were  conducted  at  pH  7.4  in  a  0.1  M  phosphate  buffer  solution  under  deoxygenated 
condition  and  all  solutions  were  prepared  with  double  deionized  distilled  water. 

Measurement  of  current  (i)  with  time  was  usually  at  about  700  mV  (vs.  SCE)  for  the  reduction  reaction 
and  at  near  2.0  mV  for  the  oxidation  reaction.  The  resulting  concentrations  of  a-keto  glutarate  and 
NH4+in  the  reaction  cell  ranged  from  2  to  300  nM  after  each  injection  unless  otherwise  stated. 


3.  Results  and  Discussion 

3.1.  Identification  of  PAMAM  Binding  with  GDH  in  Sol-Gel  Au  Colloid 

Fig.  2(a)  shows  the  UV-VIS  spectrum  of  PAMAM_2  solution  at  various  concentrations.  PAMAM’s  in 
general  have  major  absorption  peak  at  227  nm,  but  a  minor  peak  also  appears  at  278  nm.  The  peak  at 
278  disappears  at  dilute  concentration  as  shown  in  blue  in  Fig.  2(a).  GDH  solution  has  similar 
spectrum  as  PAMAM2  accept  that  its  absorptivity  is  less  intense  at  278,  as  shown  in  black  in 
Fig.  2(b).  Fig.  2(b)  also  shows  the  spectra  of  a  plastic  cuvette  (made  for  UV-VIS  range)  coated  with 
cysteamine-Au-GDH,  PAMAM_0-Au-GDH,  and  PAMAM_2-Au-GDH  in  red,  green,  and  blue, 
respectively.  The  peak  at  near  548  nm  is  associated  with  Au  particles.  The  peak  at  227  nm  would 
nearly  disappear  for  the  coating  (sol-gel  colloid)  containing  cysteamine  (red  line)  after  the  cuvette  was 
rinsed  with  deionized  water  numerous  times,  but  a  small  peak  at  278  nm  would  remain,  that  was 
believed  to  be  a  combined  cluster  layer  of  cysteamine  and  GDH  situated  in  the  Au  colloid.  The  same 
can  be  observed  if  rinsing  was  done  with  the  PAMAM’s  coating,  however,  the  remaining  cluster  peak 
at  278  nm  was  much  larger  and  the  Au  peak  at  548  nm  was  accordingly  higher.  Hence,  it  can  be 
reasoned  from  the  magnitude  of  the  absorbance  at  278  nm  that  PAMAM’s  can  bind  better  with  the  Au 
nanoparticles  and  GDH  to  form  a  colloid  attached  onto  the  electrode  surface  than  cysteamine  with  Au 
nanoparticles  and  GDH  in  this  self-assembly  process. 
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(a) 


(b) 

Fig.  2.  UV-VIS  spectra  of  PAMAM_2  and  other  senor  materials  coated  on  an  UV-VIS  cuvette:  (a)  Spectra  of 
PAMAM  2  solutions  at  various  concentrations,  characteristic  peaks  of  PAMAM’s  are  at  227  and  278  nm.  At 
lower  concentration,  as  shown  in  blue,  PAMAM  2  only  has  absorption  peak  at  227  nm;  (b)  Spectrum  of  GDH 
solution  and  spectra  of  an  UV-VIS  plastic  cuvette  coated  with  cysteamine-Au-GDH,  PAMAM_0-Au-GDH,  and 
PAMAM_2-Au-GDH  in  black,  red,  green,  and  blue,  respectively. 


3.2.  Voltammetric  Responses  of  the  Modified  GCE’s  for  NH/  Detection 

We  have  explored  several  combinations  of  sol-gel  Au  colloids  for  the  sensor  development  to  detect 
ammonia,  these  combinations  were:  Cysteamine/PAMAM_0-Au-GDH,  cysteamine/PAMAM_2-Au- 
GDH,  cysteamine/PAMAM_0-Au-GDH/NADH,  and  cysteamine/PAMAM_2-Au-GDH/NADH.  We 
have  carried  out  a  wide-range  of  ammonia  concentrations  from  2  nM  to  300  nM  with  all  the  GCE’s 
modified  with  different  sol-gel  Au  colloid  combinations  in  order  to  determine  the  champion  performer. 
Trial  measurements  of  the  electrodes  indicated  that  two  distinguish  characteristic  peaks  could  be  used 
to  relate  proportionally  the  concentrations  of  ammonia  with  heights  of  current  peaks:  a  reductive  peak 
at  near  700  mV  and  an  oxidative  peak  at  2  mV.  Fig.  3  in  the  following  is  the  voltammograms  for  the 
ammonia  measurements  generated  by  using  a  GCE  modified  with  PAMAM_2-Au-GDH.  The  blue 
voltammogram  is  measurement  of  the  blank  in  phosphate  buffer  solution.  It  should  be  noted  that  the 
reductive  peaks  at  700  mV  shifted  to  higher  positive  voltage  with  increase  of  ammonia  concentrations. 
The  oxidative  peaks  at  about  2  mV  were  also  detected  but  were  not  obviously  observed  from  the 
figure. 
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Fig.  3.  Cyclic  voltammograms  of  reaction  of  NH4+  with  a-keto  glutarate  at  pH  7.4 
for  concentrations  from  2  to  300  nM. 


Overall,  GCE  modified  with  either  PAMAM  O  or  PAMAM_2-Au-  GDH/NADH  worked  best  for  the 
detection  of  ammonia  with  a-keto  glutarate,  although  there  was  no  significant  detection  advantage  by 
using  the  more  branched  PAMAM_2  in  the  sensor  preparation.  Detection  of  NH4+  concentrations 
can  be  better  demonstrated  by  using  the  oxidative  peaks  at  2  mV.  Voltammograms  of  the  oxidative 
peaks  at  2  mV  of  the  ammonia  reaction  with  a-keto  glutarate  using  GCE  modified  by 
PAMAM_2-Au-GDH/NADH  and  cysteamine-Au-GDH  are  shown  in  Fig.  4  (a)  and  Fig.  4  (b) 
respectively. 

As  shown  in  Fig.  4(a),  there  was  about  0.45  pA  difference  between  the  blank  buffer  solution  (in  blue) 
and  the  measurement  of  2  nM  NH4+  solution  (in  brown),  which  indicated  that  the  sensor  platform  is 
capable  of  measuring  NH4+  concentration  to  subnanomolar  level.  Fig.  4(b)  is  the  voltammograms  of 
the  same  solutions,  but  current  increments  of  the  oxidative  peaks  were  relatively  small,  which  indicates 
that  the  PAMAM_2-Au-GDH/NADH  modification  makes  a  more  superior  sensor  for  low  ammonia 
concentration  detection.  The  more  efficient  of  NADH  oxidation  in  the  system  could  also  be  attributed, 
at  least  partially,  by  the  direct  binding  of  NADH  with  GDH  in  the  Au  colloid. 

Fig.  5  shows  the  amperometric  responses  measured  by  5  sensors  using  the  same  platform  with 
different  modifications  for  the  reaction  of  NH4+  with  a-keto  glutarate  at  2  mV  for  concentrations 
ranged  from  2  to  300  nM.  It  can  be  observed  that  there  were  two  near  linear  regions  that  can  be 
explored  to  be  used  for  sensor  design:  a  sensitive  region  at  concentrations  less  than  20  nM,  and  a 
relatively  less  sensitive  region  at  concentrations  larger  than  40  nM.  One  may  conclude  that  this  non- 
metal  biosensor  platform  is  ideal  for  making  sensor  for  ammonia  detection  at  extremely  low 
concentrations  (>20  nM).  It  is  unclear  at  this  time  about  the  mechanisms  why  the  modified  GCE’s 
become  less  sensitive  at  the  higher  concentration  region,  it  is  conceivable  that  there  is  a  limited 
quantity  of  Au  colloid  attached  to  the  GCE  surface  and  its  ability  to  transfer  electrons  is  saturated  and 
hence  turns  inefficient  at  higher  concentration. 
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Fig.  4.  (a)  Cyclic  voltammograms  of  PAMAM2- Au-GDH/NADH  modified  GCE  for  detection  of  NH4+  at  pH 
7.4.,  concentrations  of  NH4+  varied  from  2  to  300  nM.  The  bottom  reduction  line  in  blue  was  measurement  for  a 
blank,  (b)  Same  measurements  with  a  GCE  modified  with  cysteamine-Au-GDH. 
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Fig.  5.  Amperometric  responses  of  ammonia  reaction  with  a-keto  glutarate  from  5  different  GCE  sensors 
modified  with  similar  platform  materials:  black  is  cysteamine-Au-GDH,  red  is  cysteamine/PAMAM_0-Au- 
GDH,  green  is  PAMAM_0-Au-GDH,  blue  is  cysteamine/PAMAM_2-Au-GDH,  light  blue  is  PAMAM_2-Au- 
GDH. 
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3.2.  Reproducibility  and  Stability  of  the  Modified  GCE’s 

An  ideal  sensor  platform  should  be  able  to  generate  measurements  that  are  reproducible  and  the  system 
is  stable  with  time,  thus  the  measurements  are  consistent  and  reliable.  Fig.  6  (a)  shows  the  50  cyclic 
responses  of  a  typical  modified  GCE  in  the  same  ammonia  solution.  It  can  be  seen  that  some 
deterioration  occurred  (as  shown  by  the  characteristic  peaks  at  near  700  mV)  after  the  50  cycles  but  the 
results  are  relatively  consistent.  Fig.  6  (b)  shows  the  amperometic  responses  of  a  GCE  modified  by 
PAMAM_0-Au-GDH/NADH  at  2.0  mV  after  2  weeks  stored  inside  a  refrigerator  at  4  °C,  the 
responses  are  nearly  identified  with  the  same  trend,  the  measurements  indicated  that  the  sol-gel  Au 
colloid  stabilized  the  GDH/NADH  within  the  cluster  and  prevented  the  bioenzyme/biochemical  from 
deteriorating.  It  is  well  known  that  these  bioenzyme/biochemical  are  very  unstable,  especially  under 
dilute  concentrations  as  they  were  in  this  study.  These  results  demonstrated  that  the  biosensor  platform 
has  reasonable  reproducibility  and  stability. 
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cone,  or  ketoglutarate/plVl 

(b) 

Fig.  6.  (a)  Voltammograms  of  a  modified  GCE  with  50  measurement  cycles  of  the  same  solution; 
(b)  Amperometic  respones  of  a  GCE  modified  by  PAMAM_0-Au-GDH/NADH  at  1.8  mV, 
before  (blue)  and  after  (red)  2  weeks  stored  inside  a  refrigerator  at  4  °C. 


4.  Conclusions 

We  successfully  modified  our  glassy  carbon  electrode  biosensor  platform  [2]  for  ammonium  detection; 
we  further  modified  the  sensor  platform  with  PAMAMs  and  the  detection  lower  limit  was  enhanced. 
This  highly  modified  electrode  can  detect  ammonium  concentrations  down  to  2  nM  or  lower.  The  low 
detection  limit  of  this  biosensor  is  far  more  superior  than  most  available  detection  methods  in  the 
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public  domain.  This  biosensor  platform  can  be  modified  to  be  used  for  many  measurement 
applications  in  biomedical  and  environmental  field  that  require  high  sensitivity  at  low  concentrations. 
Currently,  the  challenge  of  biosensor  development  is  not  in  finding  the  right  enzymes  or  conjugate 
reactions  (Equation  (1))  that  govern  the  particular  species  to  be  detected,  it  is  the  portability,  stability, 
and  noise  reduction  that  demand  further  research  and  development. 
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1.  Jaiswal  A,  Wong  YYW,  Bhushan  A,  Daniels  C  &  Lai  JCK  (2010)  A  Noncontact  Co- 
Culture  Model  of  Peripheral  Neural  Cells  for  Nanotoxicity,  Tissue  Engineering  and 
Pathophysiological  Studies.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech 
2010,  June  21-25,  2010,  Anaheim,  CA  (in  Abstracts  Volume). 

Abstract.  Cell  culture  models  in  vitro  have  long  serve  as  tools  for  the  elucidation  of  the 
cellular  and  molecular  mechanisms  underlying  the  pathophysiology  of  diverse  diseases.  More 
recently  these  versatile  model  systems  have  permeated  into  the  fields  of  toxicology  and  tissue 
engineering.  Their  attraction  and  utility  for  use  in  investigations  in  toxicology  and  tissue 
engineering  appear  to  derive  from  the  fact  such  model  systems  are  particularly  suitable  for  high 
throughput  and  mechanistic  studies.  Although  these  models  do  not  exactly  simulate  the  cellular 
architecture  in  tissues  and  organs  in  vivo,  they  have  been  well  accepted  for  studies  employing  a 
single  cell  type.  Their  limitations  notwithstanding,  they  facilitate  gaining  unique  mechanistic 
insights  regarding  the  role  of  a  particular  cell  type,  a  position  not  easily  attainable  employing 
whole  organ/tissue  studies. 

On  the  other  hand,  co-culture  cell  systems  consisting  of  two  distinctly  different  but 
functionally  complementary  cell  types  provide  structural  and  functional  perspectives  that  single- 
cell-type  models  do  not  offer.  Consequently,  there  is  a  need  for  the  development  for  more  such 
model  systems  and  applied  them  to  nanotoxicological  and  tissue  engineering  studies.  Our  group 
have  been  emphasized  the  necessity  of  developing  a  variety  of  cell  model  systems  using  neural 
cells.  We  have  recently  demonstrated  some  of  the  utilities  of  such  model  systems  in 
nanotoxicological  and  tissue  engineering  studies. 

There  have  been,  however,  few  cell  model  systems  developed  employing  peripheral 
neural  cell  types  for  nanotoxicological  and  tissue  engineering  studies.  We  have  therefore 
initiated  a  systematic  development  of  this  type  of  model  systems  using  dorsal  root  ganglion 
(DRG)  neurons  and  Schwann  cells.  The  first  model  we  have  developed  involved  a  non-contact 
co-culture  model  of  Schwann  cells  and  DRG  neurons  employing  the  hanging  cell  culture  insert. 
Initially,  we  have  partially  characterized  the  DRG  neurons  and  Schwann  cells  in  single-cell-type 
cultures.  In  ongoing  studies  employing  our  new  co-culture,  non-contact  model,  we  demonstrated 
that  the  cell  survival  of  DRG  neurons  increased  when  co-cultured  with  Schwann  cells  in  this 
construct.  Thus,  the  results  from  our  studies  to  date  as  well  as  those  from  our  ongoing  studies 
demonstrate  that  our  non-contact  co-culture  model  is  highly  suitable  for  both  high  throughput 
and  mechanistic  studies  in  nanotoxicological  and  tissue  engineering  research.  Clearly  this  model 
is  also  germane  for  the  investigation  of  mechanistic  issues  associated  with  peripheral  diabetic 
neuropathy  and  other  disease  states  of  the  peripheral  nervous  system. 
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2.  Wong  YYW,  Jaiswal  AR,  Dukhande  VV,  Bhushan  A,  Leung  SW  &  Lai  JCK  (2010) 
Elucidation  of  Neuroprotective  Properties  of  Astrocytoma  (Astrocytes-like)  Cells  in 
Neural  Cell  Culture  Models  In  Vitro :  Applications  in  Tissue  Engineering  and 
Nanotoxicology.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June 
21-25,  2010,  Anaheim,  CA  (in  Abstracts  Volume). 

Abstract.  Evidence  is  accumulating  that  cell  culture  models  in  vitro  facilitate  high 
throughput  and  mechanistic  studies  in  tissue  engineering.  More  recently,  our  group  as  well  as 
other  researchers  have  developed  several  cell  models  for  the  systematic  investigation  of 
molecular  mechanisms  underlying  the  putative  cytotoxicity  of  nanoparticles  and  other 
nanomaterials.  These  and  other  studies  have  drawn  wide-spread  interest  in  the  possible 
environmental  and  occupational  health  impact  of  nanoparticles  and  other  nanomaterials. 

As  part  of  a  continuing  effort  to  further  develop  and  characterize  cell  models  for  cellular  and 
molecular  mechanistic  studies,  we  have  extended  our  previous  investigation  employing  neural 
cell  models  because  most  studies  have  traditionally  employed  peripheral  cell  types  but  not  neural 
cells. 

Astrocytes  are  known  to  protect  neurons  against  pathophysiological  assults  in  vivo  and  in 
vitro  although  the  underlying  mechanisms  are  not  fully  understood.  We  have  previously 
developed  several  cell  culture  models  in  vitro  to  systematically  investigate  several  of  the  putative 
neuroprotective  properties  of  astrocytes.  For  example,  we  have  previously  shown  U-87 
(astrocytes-like)  astrocytoma  cells  afford  SK-N-SH  (neurons-like)  neuroblastoma  cells  against 
several  pathophysiological  assaults  such  as  oxidative  stress  induced  by  depletion  of  cellular 
glutathione. 

Based  on  the  results  of  our  previous  studies,  we  hypothesized  that  different  stress  factors  can 
elicit  and/or  enhance  the  neuroprotective  effects  of  astrocytes  through  the  activation  of  astrocytic 
signaling  and  alteration  of  astroglial  function.  We  have  initiated  a  series  of  studies  employing 
our  cell  models  to  test  this  hypothesis.  The  results  of  our  ongoing  studies  demonstrate  that  U-87 
(astrocytes-like)  astrocytoma  cells  provide  SK-N-SH  (neurons-like)  neuroblastoma  cells  against 
pathophysiological  assaults  (e.g.,  oxidative  stress)  under  several  sets  of  culture  conditions  not 
previously  studied.  Other  studies  are  in  progress  to  further  elucidate  the  cellular  and  molecular 
mechanisms  underlying  these  protective  effects  proffered  by  U-87  (astrocytes-like)  astrocytoma 
cells  to  SK-N-SH  (neurons-like)  neuroblastoma  cells.  Thus,  our  results  may  have 
pathophysiological  implications  in  neuroprotection  but  also  implications  and  applications  in 
tissue  engineering  and  nanotoxicological  research. 
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3.  Leung  SW,  Gao  W,  Gu  H,  Bhushan  A  &  Lai  JCK  (2010)  Chitosan  Membrane  in 
Combinations  with  Nanoparticles  and  Adriamycin  as  a  Treatment  to  Inhibit  Glioma 
Growth  and  Migration.  NSTI  Nanotechnology  Conference  and  Expo  -  Nanotech  2010, 
June  21-25,  2010,  Anaheim,  CA  (in  Abstracts  Volume). 

Abstract.  Our  previous  and  on-going  studies  indicated  that  U87  cells  cultured  on  chitosan 
film/membrane  exhibited  significantly  slower  growth  and  proliferation  kinetics  compared  to  U87 
cells  cultured  in  the  absence  of  chitosan  film/membrane.  Chitosan  exhibits  anti-microbial 
activities  through  its  interaction(s)  with  microbial  cell  surface  thereby  altering  their  gene 
expression  and  cellular  function  and  leading  to  cell  death.  Recent  studies  of  nanometal  particles 
have  revealed  many  properties  that  were  not  previously  expected  in  biological  systems  and  thus 
can  be  explored  for  various  applications  in  biomedical  applications.  In  this  study,  we 
hypothesized  that  the  inhibitory  effect  of  chitosan  would  be  greatly  modulated  if  we  combine 
chitosan  with  nanometals  and  adriamycin,  a  common  drug  for  cancer  therapy.  Our  results 
showed  that  combinations  of  metal  nanoparticles,  adriamycin  and  chitosan  induced  cell  death  at 
different  rates,  with  reference  to  U87  cells  cultured  in  the  absence  of  chitosan  and  with  chitosan 
alone.  The  results  showed  potentials  for  pathophysiological  applications  to  inhibit  human  brain 
glioma  migration  and  invasion.  Results  of  similar  treatments  with  different  cancer  and  normal 
cells  will  also  be  presented. 


4.  Leung  SW,  Wang  Y,  Gu  H  &  Lai  JCK  (2010)  Biomedical  Applications  of  Modified 
Carbon  Glassy  Electrode  Sensor  with  Nanoparticles  and  Dendrimers.  NSTI 
Nanotechnology  Conference  and  Expo  -  Nanotech  2010,  June  21-25,  2010,  Anaheim, 
CA  (in  Abstracts  Volume). 

Abstract.  In  our  previous  reports,  we  studied  the  development  of  biosensor  platform  that  are 
capable  of  measuring  biometabolites  and  environmental  sensitive  species,  such  as  peroxide  and 
nitrate/nitrate,  to  concentration  in  the  order  of  ppb  (parts  per  billion)  or  lower. 

In  our  more  recent  development,  we  modified  our  platform  with  dendrimers  to  enhance  its 
performance.  Zero  and  second  generation  of  dendrimers  were  coated  on  the  surface  of  a  carbon 
glassy  platform  electrode  modified  with  GDH  (glutamate  dehydrogenase)  and  it  was  used  to 
measured  ammonium,  a  common  biometabolite,  at  near  neutral  pH  that  is  common  for  normal 
bioactivities. 

The  resulting  electrode  was  tested  with  ammonium  concentrations  ranged  from  0.002  to  0.3  pM 
with  satisfactory  results.  Measurements  at  lower  concentrations  had  better  resolution  than  at 
higher  concentrations  and  it  is  believed  that  the  lower  concentration  limit  can  be  better  than  the 
tested  lower  limit  at  0.002  pM.  Performance  of  the  modified  carbon  glassy  electrode  was 
compared  with  other  glassy  electrodes  that  were  modified  differently,  and  the  results  will  be 
reported. 

The  biosensor  platform  thus  far  was  proven  to  be  versatile  and  can  be  used  in  many  biomedical 
and  environmental  applications. 
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5.  Wright  GL,  Lai  JCK,  Chan  AWK,  Minski  MJ,  Lim  L  &  Leung  SW  (2010)  Metallomic 
Distribution  in  Various  Regions  of  the  Brain  as  Influenced  by  Dietary  Intakes  and  Their 
Implications.  ISEIS  2010  International  Conference  on  Environmental  Informatics, 
Beijing,  China,  August  27-29,  2010  (in  Abstracts  Volume). 

Abstract.  Lifelong  exposure  to  environmental  factors  can  influence  the  risk  of  developing 
diseases  according  to  recent  research  findings.  Environmental  stresses  ultimately  leading  to 
neuronal  cell  death  have  been  hypothesized  as  the  causes  of  the  increased  occurrence  in 
developing  Alzheimer’s  and  Parkinson’s  disease.  Our  daily  diet  is  considered  to  be  one  of  the 
most  important  environmental  factors  that  can  seriously  affect  the  development  and  proper 
functions  of  the  brain.  Depending  on  the  concentrations,  metals  and  electrolytes  can  post  some 
health  concerns,  especially  for  a  prolonged  consumption  period.  For  example,  it  was  reported 
that  excess  amounts  of  iron,  zinc  and  copper  in  the  human  brain  may  cause  oxidative  damage 
and  protein  aggregation;  the  neurotoxicity  induced  by  these  metals  may  lead  to  cerebral  and/or 
cerebellar  degeneration.  Other  reports  showed  that  there  were  differences  in  concentrations  of 
five  different  elements  (aluminum,  zinc,  copper,  manganese,  and  iron)  between  normal  human 
brain  and  brains  of  patients  with  Alzheimer’s  disease.  In  this  study,  we  investigated  30  elements, 
including  electrolytes,  and  how  dietary  intake  on  a  life-time  basis  would  affect  their 
concentrations  and  distributions  in  various  regions  of  the  rat  brain  (hypothalamus,  cerebellum, 
pons  and  medulla,  striatum,  mid-brain,  cerebral  cortex,  and  hippocampus)  and  discussed  their 
health  implications.  Information  matrices  of  these  30  different  elements  (mostly  metals)  and  their 
distributions  in  various  regions  of  the  rat  brain  were  analyzed  as  a  function  of  normal  dietary 
intake  at  different  ages  during  development.  Our  results  showed  that  metallomic  distribution  in 
various  regions  of  the  rat  brain  is  age-related.  The  results  may  help  researchers  to  identify 
possible  links  between  daily  dietary  intake  of  metals  and  electrolytes  and  diseases  associated 
with  aging  (e.g.,  Alzheimer’s  and  Parkinson’s  disease)  and  suggest  such  metallomic  distributions 
may  be  used  as  neurological  biomarkers  of  exposure  to  heavy  metals. 


6.  Leung  SW,  Siddhanti  S,  Williams  B,  Chan  AWK,  Minski  MJ,  Daniels  CK  &  Lai  JCK 
(2010)  Effects  of  Dietary  Intake  on  Metal  and  Electrolyte  Distributions  in  Various 
Organs.  ISEIS  2010  International  Conference  on  Environmental  Informatics,  Beijing, 
China,  August  27-29,  2010  (in  Abstracts  Volume). 

Abstract.  The  intake  and  concentration  of  metals  and  electrolytes  from  our  diet  are  believed  to 
be  affecting  our  general  health,  in  particular,  the  proper  functions  of  vital  organs.  For  example, 
in  addition  to  other  genetic  and  environmental  factors,  consuming  water  with  high  alkalinity  for 
prolonged  time  is  suspected  to  lead  to  diseases  such  as  kidney  stone.  Evidence  has  been 
accumulating  that  excessive  metal  intakes  would  lead  to  organ  failures.  Once  absorbed,  minerals 
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and  electrolytes  can  travel  freely  throughout  the  body,  and  distribute  at  key  organ  systems  such 
as  the  brain,  lung,  kidney,  etc.  By  conducting  experiments  with  animal  models  (e.g.,  rats),  it  is 
possible  to  not  only  determine  where  the  organ  distribution  of  various  matrices  of  elements  and 
minerals  but  also  correlate  such  matrices  with  the  overall  physiological  and  behavioral  status  of 
such  models.  In  this  study,  information  matrices  of  30  different  elements  (including  heavy 
metals  and  some  electrolytes)  and  their  distributions  in  various  vital  organs  (e.g.,  brain,  lung, 
kidney,  liver,  heart,  spleen,  and  uterus)  were  analyzed  as  a  function  of  normal  dietary  intake  at 
adulthood  (120  days  old).  An  elemental  (metal  and  electrolyte)  distribution  model  was  then 
formulated  based  on  experimental  results.  The  study  has  high  impact  to  our  understanding  of 
how  environmental  health  would  affect  our  well  being.  This  study  would  also  provide  insights 
on  how  our  diet  would  affect  the  accumulations  of  unwanted  elements,  such  as  heavy  metals,  in 
our  vital  organs.  The  results  may  help  researchers  and  health  practitioner  to  identify  possible 
links  between  daily  diet  (metals  and  electrolytes)  and  diseases,  and  may  also  lead  to  a  better 
understanding  of  diseases  associated  with  aging  such  as  Alzheimer’s  and  Parkinson’s  diseases, 
and  other  neurological  disorders. 


7.  Bhushan  A,  Patil  PP,  Bhardwaj  V,  Lai  MB,  Daniels  CK,  Leung  SW  &  Lai  JCK  (2010) 
Cross  Resistance  of  Magnesium  Oxide  Nanoparticles  to  Cisplatin  in  Leukemia  Cells: 
Mechanistic  Studies.  Institution  of  Mechanical  Engineers  Seminar  on  Nanotechnology  in 
Medicine  and  Biotechnology,  October  2010,  London,  UK  (in  Program  &  Abstracts). 

Abstract.  We  have  presented  an  overview  of  our  studies  on  anti-cancer  drug  resitance 
mechanisms  as  well  as  ongoing  systematic  studies  of  the  nanotoxicity  of  metallic  oxide 
nanoparticles.  Our  ongoing  studies  also  suggest  that  resistant  cancer  cells  may  exhibit  cross- 
resistant  to  the  cytotoxic  effects  of  metallic  oxide  nanoparticles.  The  latter  effects  appear  to  be 
mediated  by  cell  survival/proliferation  signaling  mechanisms.  Clearly  these  are  important  and 
novel  areas  of  anti-cancer  drug  discovery.  As  such  they  merit  further  investigation.  We  are 
currently  pursuing  these  exciting  new  dormains  of  nanomedicine  and  nanocancer  research. 
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